WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 6 : 

H04N 7/10, 7/14, 7/00, 5/44, H04H 1/00, 
1/02, 7/00 



Al 



(11) International Publication Number: 
(43) International Publication Date: 



WO 99/34599 

8 July 1999 (08.07.99) 



(21) International Application Number: PCT/US98/27415 

(22) International Filing Date: 22 December 1998 (22.12.98) 



(30) Priority Data: 
09/001,841 



31 December 1997 (31.12.97) US 



(71) Applicant: VSIS, INC. [US/US); 1050 East Arques Avenue, 

Sunnyvale, CA 94086 (US). 

(72) Inventor: ALLPORT, David, E.; 3832 Ross Road, Palo Alto, 

CA 94303 (US). 

(74) Agents: COHEN, Neal, M.; Lyon & Lyon, LLP, Suite 4700, 
633 West Fifth Street, Los Angeles, CA 90071-2066 (US) 
et al. 



(81) Designated States: AL, AM, AT, AU, AZ, BA, BB, BG, BR, 
BY, CA, CH, CN, CU, CZ, DE, DK. EE, ES, FT, GB, GD. 
GE, GH, GM, HR, HU, ID, IL, IN, IS, JP, KE, KG. KP. 
KR, KZ, LC. LK, LR, LS, LT, LU, LV. MD, MG, MK, 
MN, MW, MX, NO, NZ, PL, PT, RO, RU, SD, SE, SG, 
SI SK, SL, TJ, TM, TR, TT. UA, UG, UZ, VN, YU, ZW, 
ARIPO patent (GH, GM, KE, LS. MW, SD, SZ, UG, ZW), 
Eurasian patent (AM, AZ. BY, KG, KZ, MD. RU, TJ. TM). 
European patent (AT, BE, CH, CY, DE, DK, ES, FI, FR, 
GB. GR, IE, IT, LU. MC. NL. PT. SE). OAPI patent (BF. 
BJ, CF. CG, CI. CM, GA, GN, GW, ML, MR, NE. SN, 
TD, TG). 



Published 

With international search report. 



(54) Tide: SYSTEM FOR DUAl^DISPLAY INTERACTION WITH INTEGRATED TELEVISION AND INTERNET CONTENT 





(57) Abstract 

A system and method is disclosed for using two or more cooperating but physically independent displays for enhanced viewing of 
data streams, where the viewing on one display does not interfere with the viewing on the other displays. The type ;of data^ streams may be 
complex streams such as multiple television broadcast signals or other video signals, Internet data, or other local data such ^ information 
related to the state of a consumer's home appliances. The signals may be data embedded in the vertical blanking interval (VBI) for analog 
signals, or in dedicated slots in a digital signal according to protocols such as MPEG. The system includes a haiiMddan d portable remote 
control (10) with a motion picture display and hardware or software enabling interaction between a primary display (80) and the hand-held 
display (10). The hardware may include an integrated TV tuner or a physically separate base station (75) having a TV tuner, in wh.ch case 
the base station (75) and the remote control (10) would communicate via radio frequencies (110 and 115). 
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DESCRIPTION 

System For Dual-Display Interaction With Integrated 
Television And Internet Content 

Related Applications 

This application contains subject matter related to co- 
pending U.S. patent application filed under attorney docket 
number 228/269, entitled "Portable Internet-Enabled 
Controller and Information Browser For Consumer Devices," 
filed December 31, 1997, invented by the same inventor as 
the present invention, and to be assigned to the same 
assignee as the present invention. The entire original 
disclosure of said co-pending application is herein 
expressly incorporated by reference. 

Terminology 

As used in this application, the term "data stream" 
refers generally to transmitted information, and a data 
stream may itself contain multiple data streams. A data 
stream may be HTML data transmitted from the internet, or it 
may be a "media stream" such as an analog or digital TV 
broadcast signal, satellite TV signal, cable TV signal, or 
other audio and/or video signal. Media streams such as TV 
broadcast signals may contain several channels, and each 
channel may further contain audio, video, or other embedded 
data streams, including HTML data. Furthermore, HTML data 
is used to refer to any internet-derived data, as opposed to 
solely data that is in the HTML protocol format, as the 
concepts described herein are equally applicable to other 
internet-derived data, especially data that is transmitted 
in a widely accepted protocol. 

Field Of The Invention 

The present invention relates generally to systems and 
methods for using two or more cooperating but physically 
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independent displays for enhanced viewing of data streams on 
each display, wherein the viewing of a data stream on one 
display does not interfere with the viewing of a data stream 
on the other displays. More particularly, the images may be 
derived from complex data streams such as multiple TV 
(television) broadcast signals (and/or cable TV or satellite 
transmissions or other video signals), internet data, and/or 
other local data such as information related to the state of 
a consumer's home appliances. In addition, the different 
displays may be available for viewing by more than one 
person. 

Background Of The Invention 

As technology rapidly advances, there are more and more 
sources and types of information available to a consumer. 
For example, after TVs improved from simple black-and-white 
TVs to color TVs, cable and satellite TV became available. 
Most recently, internet-enabled TV became available to 
provide both TV and internet content. In a fast-paced 
society such as ours, consumers are looking for ways to 
enhance the overall viewing experience and to absorb more of 
the available information from various sources, in less 
time . 

One of the simplest methods of obtaining more 
information is to use multiple independent displays, such as 
a having two TVs tuned to two separate channels, or two 
closed circuit cameras (e.g., security cameras) focussed on 
separate areas. The problem with this method is that the 
person viewing the information may have to use various non- 
cooperating controllers for controlling the different 
devices, and may even have to physically change locations 
after viewing one display in order to be able to view 
another display. Additionally, this type of method usually 
involves fixed displays, as opposed to at least one portable 
display. And though it is possible to use a single 
universal remote control to control both TVs, additional 
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problems exist including the cost of multiple TVs or 
monitors, the physical space required for the TVs, and the 
availability of other electronics hardware such as cable 
receivers, closed circuit wiring, etc. 
5 Other multiple-display systems have displays in close 

physical proximity to each other and may display related 
data, but if integration of the data is to occur, it must be 
accomplished manually, as the underlying systems are not 
capable of doing so. That is because the physical display 

10 devices are each dedicated to providing a limited amount of 
information related to a limited topic of interest. 
Examples of these types of systems include radar screens and 
controls in a cockpit of an airplane, and medical 
instruments monitoring various vital signs of a patient in a 

15 hospital. 

One method of enhancing the viewing experience involves 
using multiple displays to simultaneously display multiple 
duplicate data streams. This is common in electronics 
stores where many TVs are all tuned to the same channel, or 

20 in exercise classes where many TVs are all playing the same 
exercise video tape. Though perception of the data stream 
(i.e., TV program or video tape) might be enhanced in such a 
case, the viewer is only viewing multiple copies of the same 
data stream, and is not benefitted with access to data 

25 streams of differing content. 

Other multiple-display systems display separate parts 
of a single image, and the displays taken as a whole are 
designed to represent the desired image. The displays are 
arranged in a predetermined pattern such as side-by-side, in 

30 a matrix, or even to form a cylindrical "circle vision" 
theater. Again, though the viewing experience is altered 
and perhaps enhanced, the viewer is only viewing what is 
designed to be a single image from multiple data streams. 
In fact, these systems may even . detract from the desired 

35 image perception due to the gaps between the several display 
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screens and/or the imperfect placement of the various parts 
of the overall image within the various display screens. 

Another attempt to provide consumers with a method of 
viewing multiple data streams simultaneously to obtain more 
information, is through the use of windows-type operating 
systems used with PCs. The user may swap between multiple 
windows to view various data streams. However, these 
systems are designed under the assumption that the viewer 
concentrates on a single type of application at a time 
Thus, the same physical display is used for the multiple 
windows that represent various applications. A particular 
window may be enlarged to cover the entire display, but then 
the benefit is lost of having the capability to view 
multiple data streams substantially simultaneously. 
15 Furthermore, the individual applications are typically 
independent of each other, and do not communicate with each 
other except for perhaps during data transfer. Even where 
multiple windows are viewed simultaneously, such as during 
video conferencing where a person may view multiple 
20 conference "attendees" in various windows, the images of the 
attendees all appear on the same physical display, and the 
more attendees there are, the less room there is for each to 
be displayed. 

Various systems also exist that attempt to solve the 
25 problem of viewing multiple data streams substantially 
simultaneously with two or more cooperating displays. A 
common system is a pip (picture-in-picture) system, which 
allows simultaneous viewing of two TV programs at once, for 
example. Though the viewer may "channel surf" a first data 
stream without affecting the overall image produced by the 
second data stream, both data streams are displayed on the 
same physical display, and therefore there is some 
degradation of the viewing experience of each. For example, 
a primary program may be playing on the entire physical TV 
display while a secondary program (the PIP) is playing in a 
small area in the corner of the display. The primary 
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program is degraded due to the screen real estate used up by 
the secondary program, and the secondary program is degraded 
because it is not able to occupy the full size of the 
physical display. 
5 Similarly, existing TV interfaces provide a wide 

variety of available functions such as data display (e.g., 
to adjust attributes of the TV and related devices) f 
advertisements, internet browsing, alternate channel 
viewing, etc. But the additional functions are all managed 

10 by assigning different amounts of the same physical screen 
to the additional function or functions. This approach of 
sharing the same physical screen real estate for all 
operations performed by a device means that combined 
activities restrict the display potential of each other, 

15 unless each activity occurs serially, which would defeat the 
entire purpose of allowing simultaneous viewing of multiple 
data streams in the first place. Moreover, the resolution 
of standard TV display formats such as NTSC (the American 
standard) and PAL (the British standard) are not very 

20 suitable for displaying static text. 

Another type of system that involves data integration 
to some extent is a computer network, or even simply a 
remote PC communicating with a host. Software synchronizes 
the data files stored on each device, and files can then be 

25 downloaded from the host to the remote PC, edited remotely 
on the PC, and then transferred back to the host. However, 
this setup is designed primarily for one or the other device 
to be used independently. The simultaneous use of both 
devices occurs only during file transfer, or in some cases 

30 to achieve the same type of extended screen real estate for 
a single-image display that occurs with two or more displays 
side-by-side . 

Thus it would be desirable to provide systems and 
methods for using two or more cooperating but physically 
35 independent displays for achieving enhanced viewing and/or 
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browsing of data on each display, without affecting the 
viewing and/or browsing of data on the other displays. 

Summary Of The Invention 

, h ™ S PreS6nt inVention i» based upon the recognition 
that there are many situations in which users wish to view 

Z b ST™',** PerCe±Ve m ° re than ° ne data ^ream 
substantially simultaneously, without having the experience 

of any data stream degraded by the others. A hand-held, 

portable remote control, with an integrated video display 

capable of displaying full motion video is thus disclosed, 

xn combination with hardware and/or software that enables 

andTrT !T een ^ ° r ° ther Pr±mary 

and the hand-held display. The hardware may include an 
integrated TV tuner and/or various data communication ports, 
or a physically separate base station with a TV tuner and/or 
various data communication ports. m the latter case, the 
base station and the remote control would communicate 
preferably via RF (radio frequency) . 

Many applications exist where the user or users may 
derive a great benefit from an additional display screen for 
user interaction, as will be discussed herein. One such 
situation is for users of TV systems that display text-based 
status information on the TV display during attribute 
adjustment (e.g., to volume, contrast, color, tint, 
brightness, sound, etc.,, or for closed caption messages 

cont iVT tS SC ° reS ' St ° Ck qUOt6S ' etC " The 
control display may be used to display that information and 

therefore leave the primary viewing screen free 
unnecessary clutter. For existing internet-enabled TV the 

/ eali2ed ±S 6Ven 9reater dUe t0 the -*nificant 
amount of screen real estate typically devoted to displaying 



35 



navigational and other information. By adding a second 
display that communicates with the TV, all C f the 
navigational interface can be removed from the TV screen 
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Video games (both single user and multi-user) are 
another application where the users may realize a benefit 
from an additional display. Many computer games exist that 
allow multiple players in different locations to play with 
5 (or against) each other. These often take the form of 
interaction in on-line "virtual worlds." In order to play 
these games effectively, a user must be aware of the general 
topology of the virtual world, as well as where he or she is 
in the world in relation to others. The interface to the 

10 virtual world may be presented in various perspectives, and 
from various distances. In any case, the user's sense of 
where he or she is in the world can be greatly enhanced by 
the provision of an additional view. 

Similar advantages apply to other types of virtual 

15 world interactions, such as three-dimensional multi-user 
chat rooms, where the present invention would enable 
simultaneous first-person and alternative more global views 
from selected vantage points. This will greatly assist the 
navigation through the virtual world, and also increase the 

20 perceived immersion in the world by avoiding constant 
context-switching on a single screen. 

Another application of the present invention finds use 
in working with internet-enabled TV. Several companies are 
working on (or are marketing) technology that integrates TV 

25 and internet content within the same signal. For analog 
signals such as TV broadcasts, internet data is embedded 
within the VBI (vertical blanking interval) . Examples of 
this are Intel's "Intercast" standard (which some TV 
networks are already using), and Navio's HTML-TV standard. 

30 Likewise, Thomson Sun Interactive has been using an "Open 
TV" standard in Europe, in which the technology for both 
broadcasting and receiving signals with data embedded using 
the Open TV standard has been implemented. Another company ■, 
Starsight, markets hardware and software capable of 

35 receiving and decoding signals with embedded data, where the 
hardware and software is for use with third party units. 
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Data may also be embedded in digital signals using the MPEG 
standard. The embedded data typically may be displayed as 
text and graphics independently of the image displayed from 
the normal video signal. 

The present invention would allow both the regular 
signal, and the embedded data, to be viewed simultaneously 
on different displays - something that is not possible using 
the technology provided by the aforementioned systems. This 
not only provides clearer viewing of both data streams due 
to elimination of screen clutter, but also allows "semi- 
synchronous" interactions with the TV and internet data, as 
described herein. 

The present invention is especially useful in family or 
group situations where multiple people are viewing the same 
15 TV screen simultaneously. One person could channel surf 
actual TV content on the hand-held remote control, while the 
other viewers enjoy the TV program on the primary display 
(e.g., the TV screen). This effectively turns a legacy TV 
into a TV with pip capability, with the added benefit that 
the PIP is actually on a separate display. Using suitable 
communications infrastructure, which is described in more 
detail herein, the remote control's display may also 
function as a regular portable TV. 

Additional uses of the present invention may be for 
video-conferencing, wherein each of many displays may be 
used, to show one attendee of the conference, or for 
controlling other various consumer devices and appliances 
such as those that respond to IR (infrared) commands, as 
disclosed in previously-referenced co-pending application 
[attorney docket #228269] . The full motion video capability 
of the remote control allows the user to view the output 
from security cameras, a video baby monitor, etc., or to use 
the remote control and its display to form part of a video 
phone system (with the addition of microphones), all without 
interrupting the primary display screen such as the TV 
display. And in an embodiment where the remote control has 
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enough memory or access to an . outside storage device, the 
remote control may also function as a convenient front-end 
to DVD (digital versatile disk) players and recorders, 
allowing the user to browse actual video, audio, and other 
5 recordings to select a desired title for playing. 

Accordingly, it. is an object of the present, invention 
to provide improved systems and methods for providing an 
enhanced overall viewing experience of multiple data 
streams . 

10 It is a further object to provide improved systems and 

methods for allowing substantially simultaneous viewing of 
multiple data streams. 

It is a further object to allow substantially 
simultaneous viewing of multiple data streams using two or 
15 more cooperating but physically independent displays. 

It is a further object to provide a hand-held remote 
control with an integrated video display for providing the 
means by which one of the data streams is viewed. 

Other objects and advantages of the present invention 
20 will become ' apparent from the detailed description of 
preferred embodiments which follow, when read in conjunction 
with the drawings. 

Brief Description Of The Drawings 

FIG. 1 shows a preferred embodiment of the remote 
25 control of the present invention with a variety of physical 

actuating buttons, and a blank display. 

FIG. 2 is a system diagram of a preferred embodiment of 

the present invention, showing the channels of communication 

between the various system components . 
30 FIG. 3 is a high-level schematic of hardware used in a 

preferred embodiment of a base • station of the present 

invention that is used in conjunction with a remote control 

of the present invention. 
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FIG. 4 is a high-level schematic of hardware used in a 
preferred embodiment of a remote control of the present 
invention. 

Detailed Description 

The present invention will be described as a "dual 
display" system, but it is to be understood that the 
concepts described herein are equally applicable to a 
"multiple display" system having three or more cooperating 
displays. Of course, implementation of the latter system 
would require more hardware and software for managing the 
communications between the various displays. 

Before describing the operation of the system and 
methods of the present invention, it will be helpful to 
describe the physical appearance of a preferred embodiment 
of a remote control 10 that forms part of the system. 
Turning to FIG. 1, the drawing is not to scale, but 
preferably the remote control 10 is a hand-held size on the 
order of only several inches long by several inches wide by 
a few inches deep, such that it may be carried comfortably 
in one hand or in a shirt pocket. 

The display area 15 is surrounded by various physical 
actuating buttons 20, 25, 30, 35, 40, 45, 50, 55, and 60. 
There may be, for example, up to twenty physical buttons, or 
less than twenty, and more preferably less than fifteen, and 
more preferably less than twelve, and still more preferably 
ten or less. The display area 15 preferably comprises an 
LCD and is preferably 320 pixels by 240 pixels {H VGA) . It 
may- have higher resolution (i.e., greater than 320 pixels 
and/or greater than 240 pixels) or lower resolution, but 
preferably not lower. It may be monochrome or color, and is 
preferably touch sensitive. It must be capable of 

displaying full motion video, such as the images produced by 
broadcast TV signals. 

The physical actuating buttons may include push buttons 
(e.g., 20, 30, 35, 40, 45, and 50), switches (e.g., 25), 
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sliders (e.g., 55 and 60 which slide in slots 65 and 70 
respectively), or other mechanisms not shown such as touch- 
sensitive pads, levers, toggles, dials, knobs, pull buttons, 
track balls, a mouse, etc. The functions of the buttons may 
5 vary, and some may have fixed functions while others may be 
programmable. Common functions may include a power on/off, 
a help button, and a mode-switching button wherein the 
remote control 10 is switched from a viewing mode to a 
control mode. The display 15 may also have some 

10 programmable buttons (also known as "soft keys)" in certain 
modes of operation. A more detailed explanation of the 
possible operation and function of the buttons is set forth 
in previously-referenced co-pending U.S. patent application 
[attorney docket #228/269] . In addition, further uses may 

15 be made of the video capability of the remote control 10 
here, such as by using video clips during configuration 
and/or during "help" mode. 

The remote control 10 is used in combination with 
hardware and/or software to form a system that enables 

20 interaction between the TV or other primary display, and the 
display 15 of the remote control 10. The applications of 
the system as previously discussed are described here in 
slightly more detail, and a description of the hardware 
and/or software that enables the system to be used as herein 

25 described will follow. 

For the situation where an internet-enabled TV display 
is cluttered with text-based and other navigational 
information, a further benefit of the present invention is 
that with suitable HTML (Hypertext Markup Language) parsing 

30 software 10, some parts of the actual internet content could 
also be displayed on the remote control's display 15. This 
frees up even more of the primary display for actual 
content. For example, many web sites employ "frames" with 
their own navigational aides in the form of graphical icons, 

35 indexes to sub-pages, etc. These could be displayed on the 
hand-held remote control, again leaving more room for the 
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image created by the primary data stream to be shown on the 
TV screen. Some of the buttons on the remote control 10 in 
this context could be used for bookmarking web sites,' 
context switching (e.g., from browsing the web to watching a 
TV program), internet navigation, etc. 

Moreover, for both internet-enabled TV and regular TV, 
an additional benefit to be realized is that the resolution 
of the text-based information on the remote control display 
15 could be much greater than that on the TV displays 
because the remote control display 15 may have an LCD 
(liquid crystal display) as opposed to a CRT display, and 
therefore may employ VGA, SVGA or better technology as 
opposed to NTSC or PAL. The function of one of the buttons 
on the remote control 10 in this application may be to swap 
the images shown on the displays. other functions for which 
sliders 55 and 60 are particularly suited could be used to 
change channels on the TV and on the remote control ■ s 
display 15 respectively. 

During virtual world communications, such as in virtual 
world games (single or multiple user), the benefit of the 
present invention is that the interface to the virtual world 
may be presented in various formats on the two separate 
displays. The player is thus provided with more strategic 
information. Various perspectives could be a first person 
view, wherein the virtual camera takes the viewpoint of a 
character in the world that is being played by the user; a 
second person view, wherein the virtual camera takes the 
viewpoint of a character in the world that is being played 
by a different user; or a third person view, wherein the 
virtual camera takes the viewpoint of no particular 
character, but instead has a global view of the world, 
usually focussing on the character being played by the user. 
The two simultaneous views may be from different 
perspectives, or they may be from the same perspective, with 
one view displaying a close-up while the other displays a 
"bird's eye" view, or any other combination. Here too, the 
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function of one of the buttons on ■ the remote control 10 may 
be to swap the- images shown on the displays. Or one 
function may be to enlarge or reduce a particular display, 
or to cycle through the available displays so that a player 
5 may perceive the world as ■ one of his or her "enemies" or 
"allies" is perceiving the world. Other functions may be 
for other game controls, which may also be performed via an 
IR keyboard or other. input means. 

When using the present invention with internet-enabled 

10 TV, an additional perceived benefit is in the use of "semi- 
synchronous" interaction with TV ads. TV ads on internet- 
enabled TV typically provide addresses or links to web 
sites, where a viewer could obtain more detailed information 
and review the information at their own pace. With the 

15 present invention, a viewer could use the remote control 10 
to bookmark web sites from advertisements as they occur or 
soon thereafter, and then later use the remote control 10 to 
contact the web sites for further information. The viewer 
could even do so during the time when regular programming 

20 (non-ads) returned to the TV, and browse the internet sites 
on the remote control's display 15 without affecting or 
interfering with the primary TV display. Furthermore, 
existing internet-enabled TV systems typically require the 
use of navigational input means (such as a mouse or IR 

25 keyboard) to access specified functions or to enable 
internet links appearing on the display. With the present 
invention, the same type of navigational aides could be used 
for accessing information on the remote control's display 
15. However, it is possible, and desirable, for such* access 

30 to be accomplished directly through the use of buttons, thus 
preventing the need for excess input and navigation. 

The present invention also provides a benefit for the 
situation where additional data (e.g., HTML data) " is 
embedded in the VBI of the TV signal, or in dedicated slots 

35 within a digital signal. That data could be stripped from 
the signal and viewed on the remote control's display 15 
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while the regular video signal appeared on the primary 
display. This differs from how current internet-enabled TVs 
make use of VBI data, because they all operate under the 
presumption that only one display is available for both the 
regular video signal and the VBI data. Some of the buttons 
on the remote control 10 in this context could be used for 
changing font sizes of any text data, scrolling through 
text, or swapping the programs showed on the two displays. 

An especially useful application of the present 
invention is in the context of family or group TV viewing is 
the ability to channel surf. This may also be useful for a 
single viewer. A viewer may channel surf for alternate 
programs of interest, without interfering with the program 
then playing on the TV. The advantages of being able to see 
alternative channel program listings on a hand-held display 
separate from the main viewing screen are described in the 
previously referenced co-pending application [attorney 
docket #228/269]. The present invention also enables the 
user to actually select the channel, and view the alternate 
program on the remote control's display 15. Headphones may 
be used to receive the audio portion of the alternate 
channels being surfed. After finding a desired alternative, 
a user may execute a command (by the press of a button for 
example) to cause the primary viewing screen to display the 
desired program that was found while surfing. m addition, 
a button may allow the user to execute a "swap" such that 
not only does the primary display switch to show the desired 
program that was found while surfing, but the hand-held 
display 15 switches to show the program that was being 
played on the primary display as well. 

Turning now to the hardware and/or software of the 
present invention, it is to be understood that the drawings 
are of preferred embodiments only, and that other 
configurations may be designed without varying from the 
spirit and scope of the inventive concepts described herein. 
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The software may be written in any convenient 
programming language or languages, and preferably comprises 
an underlying system control component and an application 
interface component . The system component controls the 
5 execution of the actual commands necessary, to control the 
consumer devices. It controls the storage, access, and 
other data management tasks necessary for proper 
functioning. It also retains an internal representation of 
each device being controlled, its type {CD player, TV, 

10 etc.)/ name, and associated icon if applicable. The system 
component also maintains a table of IR commands for each 
device, along with associated data such as the 
representation of the commands on the remote control ' s 
display,, if applicable. The application interface component 

15 acts as a front-end for the consumer to program and 
otherwise make use of the system component, and is 
implemented such that a control algorithm allows - the 
consumer to navigate through the available screens to 
accomplish the desired control over consumer devices. 

20 The software is preferably programmable by the user and 

presents a unified system view of various devices being 
controlled. For example, there may be volume and channel 
controls on one screen which control the volume of a stereo 
speaker connected to a TV output and the channel on a cable 

25 box connected to the TV input respectively. The ease of 
this system contrasts with that of known universal remote 
controls where it is necessary to first select a "stereo" 
mode, then change volume, then select "cable" mode to change 
channel. 

30 The programming may be accomplished with an integrated 

graphical keyboard for simple tasks such as entering and 
modifying passwords, or for more complex programming, but 
the latter is preferably done with the use of companion 
software either using an IR keyboard or other input means in 

35 conjunction with the display 15," or by using PC software on 
a PC. For example, the functions to be performed by certain 
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buttons may be programmed by companion software, and the 
display may then show the association of physical or touch 
screen actuating buttons with the actual functions they 
perform. Passwords and filters may be programmed to assist 
parental control or other monitoring, and screen layouts may 
also be designed. For a more detailed description of 
associated software, the previously mentioned co-pending 
U.S. patent application [attorney docket # 228269] may be 
consulted. 

Turning now to the hardware, FIG . 2 shows a system 
diagram of how components of the present invention cooperate 
with existing technology to implement the concepts described 
herein. The hardware for the base station 75 will be 
described in more detail in conjunction with the description 
of FIG. 3, and the hardware for the remote control 10 will 
be described in more detail in conjunction with a 
description of FIG. 4. 

FIG. 2 shows the base station 75 as a physically 
separate component from the remote control 10, which is a 
preferred embodiment, but the base station 75 may be 
implemented as an integrated part of the remote control 10 
or of the TV 80 if desired. In fact, some existing ATVs 
(advanced TVs) are equipped with a two-way internet 
connection, a decoder for separating out embedded data from 
analog or digital signals, and hardware and software to send 
complex data streams including video signals. For users 
with ATVs such as this, the remote control 10 needs only to 
have the capability of receiving and displaying complex data 
streams, and needs no additional communication ports other 
than a control IR port for sending IR commands. In such an 
embodiment, the elimination of additional communication 
ports and/or a decoder may have a significant effect on the 
cost of the remote control 10. 

Other advantages of having a base station 75 separate 
from the remote control 10 (i.e., either physically 
independent or integrated in the TV 80) are that the remote 
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control 10 may be lighter in weight and smaller in size. 
And most significant, the remote control 10 could operate in 
a wireless mode, with the base station 75 having any needed 
wired connections to the home cable box, satellite TV 
5 receiver, VCR, etc. Size, weight, and the ability to 
operate in a wireless mode are critical features in 
providing an overall enhanced viewing experience. 

Still referring to FIG. 2, various signals (data 
streams) 85 from various sources enter the base station 75 

10 as indicated by arrow 90. The signals 85 may originate from 
broadcast TV, cable TV, satellite TV, a VCR, a laser disc, a 
DVD, or any other source, they may be analog or digital, and 
they may contain embedded data as previously described. 
Other data 95 from an outside data source such as the 

15 internet may enter the base station 75 as indicated by arrow 
100. In practice, it is preferable, as indicated by FIG. 2, 
that all data streams enter the base station 75, which may 
then act as a switching station for the data streams by 
determining whether to send the data streams (or parts 

20 thereof) to the TV 80, to the remote control 10, to both, or 
to neither. This would be extra beneficial, for example, if 
the TV 80 had only one input, because instead of having to 
physically switch cables to change from one input to 
another, the switching could be accomplished within the base 

25 station 75 and the base station 75 could send the proper 
input signal to the TV 80. However, it is technically 
feasible to have incoming signals sent both- to the base 
station 75 and to the TV 80. 

Data streams pass from the base station 75 to the TV 80 

30 by wired connections as indicated by arrow 105. There may 
be only one connection, or there may be many. A typical 
. connection may include a standard TV cable which passes all 
or part of data streams 85 to the TV 80. Other connection 
means (such as RS232, USB, and/or modem ports or other 

35 digital I/O ports for DVD media, etc.) . may exist in the TV 
80 for passing all or part of data streams 95 to the TV. 
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The latter connections would apply, for example, to ATVs 
with internet capability. 

Data streams pass from the base station 75 to the 
remote control 10 by wired or wireless communications, 
5 indicated by arrow 110. Wired communications may employ 
either USB or Firewire 1394 protocols, both of which are 
capable of transmitting full motion video. However, wired 
communications are not preferred, as they detract from the 
overall viewing experience by inhibiting the user's 
10 mobility. 

For wireless communications, the base station 75 may 
transmit data to the remote control 10 by way of fast IrDA 
or RF, but the preferred method is RF, in .which case the 
frequency of transmission would be preferably at 900MHz, 
2.4GHz, or other FCC-approved home communications 
frequencies. The main reason RF is preferred is that an 
important aspect of the present invention is the ability to 
v le w video on the remote control's display 15. For the base 
statxon 75 to efficiently and accurately send video signals 
at an acceptable rate of speed, IR is not as well-suited as 
RF. IR transmissions have limited range capabilities, and 
are useful typically only for sending small amounts of data 
not the large amounts of data in a video signal. Even "fast 
IrDA" technology, which is capable of transmitting video 
signals, can operate effectively only within about one meter 
at most. Furthermore, IR signals will not operate through 
walls, whereas RF signals will, thus with RF communications 
a user may operate the remote control 10 from a room 
Physically separate from the room where the base, station 75 
is located. 

Communications from the remote control 10 to the TV 80 
(as indicated by arrow 120) are preferably by standard 
control IR. Typical communications from the remote control 
10 to the TV 80 may be to adjust TV attributes such as 
volume, channel, input, etc. The remote control 10 may also 
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send commands to other devices/ such as a DVD player, VCR, 
etc., for the control thereof. 

Communications from the remote control 10 to the base 
station 75 (as indicated by arrow 115) are also preferably 
5 by standard control IR, though RF communications to the base 
station 75 could be an option because some of the hardware 
would already be present due to the need for the base 
station 75 to transmit by RF to the remote control 10, as 
previously described. Typical commands from the remote 

10 control 10 to the base station 75 may be a request for a new 
channel to view on the display 15 (i.e., channel surfing), a 
request to swap displayed programs with the TV 80, a request 
to access new data from the internet or other outside data 
source, or any other request to the base station 75 for data 

15 streams. 

The type of request from the remote control 10 to the 
base station 75 will vary greatly depending on the 
application in progress. For example, during a video game, 
a request may be to provide another view of the virtual 

20 world topology. The base station 75 may satisfy that type 
of request by retrieving the data 95 from an outside source 
such as the internet, or alternatively the base station 75 
may do processing internally to provide the other view. 
Other types of requests may be to access a new web site 

25 while browsing the internet, in which case the base station 
75 would access the data 95 and transmit it to the remote 
control 10 . 

An extremely useful feature is to have the data streams 
on the TV's display and on the remote control's display 15 

30 swapped. In order for this to occur, the base station 75 
that is providing a channel to the remote control 10 must 
tune to a new channel (the one the TV is tuned to just prior 
to the swap), and the TV must tune to a new channel (the one 
the base station is tuned to just prior to the swap) . 

35 Therefore, there needs to be a way for the system (e.g., as 
seen in FIG 2) to know what channel the TV 80 is tuned to 
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just prior to the swap, and what channel the base station 75 
is tuned to just prior to the swap. A preferred method, 
though by no means the only method, is for both the base 
station 75 and the TV 80 to get the required information 
from the remote control 10. This will usually be possible 
because the remote control 10 is aware (i.e., may store in 
its memory) of the channel the TV is tuned to based on the 
fact that the remote control 10 will have sent the "change 
channel" IR command to the TV 80 in the first place. 
Similarly, the remote control 10 will be aware of the 
channel the base station 75 is tuned to based on the fact 
that the remote control 10 will have sent the "change 
channel" IR command to the base station 75 in the first 
place too. Thus, the remote control 10 may be programmed to 
send IR control commands to both the TV 80 and to the base 
station 75 with the push of a single button. 

So if the user wants to swap the programs being 
displayed on the TV 80 and the remote control's display 15, 
a single button on the remote control may be programmed to 
execute the swap. An example of the sequence of commands 
sent by actuating the swap button is as follows. First, a 
command is sent from the remote control 10 to the base 
station 75 to tune the base station's tuner to the channel 
at which the TV 80 is tuned. As previously stated, the 
remote control 10 already knows the channel.. That would 
complete the first half of the swap, as the program then 
being displayed on the TV 80 would be displayed on the 
remote control's display 15 due to the base station 75 
tuning to the proper channel. Note that the remote control 
10 will have known the channel that the base station 75 was 
tuned to just prior to the swap as previously stated, and 
would have stored that information in its memory for 
subsequent use to complete the swap. That information is 
needed because next, a command is sent from the remote 
control 10 to the. TV 80 to tune the TV tuner to that 
channel, i.e., the channel that the base station 75 was 
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tuned to just prior to the swap. The swap is now completed, 
as the TV would then be tuned to the channel that the base 
station 75 was tuned to just prior to the swap, and would 
therefore be displaying the program that was being displayed 
5 on the remote control's display just prior to the swap. The 
remote control 10, of course, would retain in its memory the 
channels which it commanded both the base station 7 5 and the 
TV 80 to tune to, so that the process could be repeated. 

Adding a slight twist to the previous example, assume 

10 the user is viewing channel 1 on the TV display, and channel 
2 on the remote control 1 s display 15. In that situation, 
the TV 80 is tuned to channel 1. The base station's input 
signal 85 is from broadcast TV, and its tuner is set to 
channel 2 . Now, the user wants to watch a video on the 

15 remote control's display 15. The user will send a command 
to the base station 75 to switch its input 85 from broadcast 
TV to the VCR, and a command to the VCR to begin playing. 
These commands could of course be sent by actuating a single 
button on the remote control 10. Now, if the user wants to 

20 execute a swap so that the TV 80 is displaying the video, 
and the remote control 10 is displaying channel 1, the 
following is a typical sequence of commands that would be 
sent. First, a command sequence is sent from the remote 
control 10 to the base station 75 for the base station 75 to 

25 switch its input to broadcast TV, and to tune its tuner to 
channel 1. The first half of the swap is complete. Next, a 
command is sent to the base station 75 to output from the 
switching and MUX box 265 the VCR signal, instead of the 
broadcast TV signal. Next, a command is sent to the TV to 

30 change to channel 3 (or whatever the appropriate channel is 
for VCR input) . The second half of the swap is now 
complete. The TV 80 would be showing the video, and the 
remote control 10 would be showing channel 1. Again, the 
remote control 10 could be programmed such that these 

35 commands could be sent by actuating a single button on the 
remote control 10. 
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For applications where embedded data is to be separated 
from the signal 90 or other data 95, the remote control 10 
may request that the embedded data be sent to itself while 
the main portion of signal 90 or data 95 is transmitted to 
the TV 80. For example, a broadcast TV signal 85 may 
contain VBI-embedded data that is related to the program 
being broadcast, such as biographies of the actors, 
information about upcoming episodes, historical facts, etc. 
The user could then browse the VBI-embedded data on the 
remote control 10 while viewing the program on the primary 
viewing screen of the TV 80. If the VBI-embedded data (or 
digitally embedded data from a digital signal) is HTML data, 
the remote control 10 may display options for the user to 
respond to program-related issues, for example by voting on 
what should happen to certain characters, or whether the 
episode was good or bad. The user could then respond by 
using the appropriate buttons on the remote control 10 (as 
indicated on the display 15), which would be programmed to 
send appropriate information to an associated web site. Or 
the embedded data may actually be a web page providing 
access to a web site related to the program being broadcast, 
in which case the remote control 10 could access the web 
site through the base station 75 to gain more detailed 
information related to the program. Of course, the embedded 
data does not have to be related to the broadcast program, 
but a likely use of the embedded data is that it would be so 
related. Another use for embedded data is to send lists of 
web sites associated with advertisements being broadcast 
The user could then browse the list on the remote control's 
display 15 during or after the advertisements, and tag or 
bookmark them for later reference. There are many other 
uses of embedded data, and the foregoing are intended to be 
only illustrative, and in no way limiting of the types of 
data the present invention is designed to make use of. 

The previous discussions have presumed for the most 
part that the base station 75 was physically separated from 
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the TV 80 and from the remote control 10. It should be 
appreciated that in an embodiment where the base station 75 
is physically integrated with the TV 80, the associated 
communication channel (indicated by arrow 105) would not be 
5 required. Similarly, if the base station 75 is physically 
integrated with the remote control 10, there is no need for 
the base station 75 to "send" video signals to the remote 
control 10, and the associated communication channels 
(indicated by arrows 110 and 115) would not be required. In 

10 the latter case, the RF communications hardware could then 
be eliminated, producing additional cost savings, size 
reduction, maintenance benefits, etc. 

The hardware of a typical base station 75 will now be 
described, as seen in FIG. 3. The hardware is capable of 

15 receiving various data streams (signals 85 and/or other data 
95) , analog and/or digital, and of RF encoding the data 
streams and transmitting them for receipt by an RF receiver 
(e.g., one in the remote control 10). The hardware is also 
capable of processing and passing along data streams to both 

20 analog TV inputs and digital TV inputs as indicated by 
arrows 255 and 2 60 respectively. 

Signals 85 may be analog or digital. Analog signals 
will enter the base station's "tuner + analog to digital 
converter" 125, and "analog switching and MUX box" 2 65. 

25 Digital signals enter the base station 75 through USB port 
.130, or through other digital . inputs capable of handling 
higher-bandwidth data. Other data 95 (including HTML data) 
may enter the base station 75 through various ports 130, 
140, and 145, as is described herein, and even through IrDA 

30 port 135, though that is not preferred. The incoming data 
85 and/or 95 may be transferred to the data bus 150, but 
that is not necessary, as data passing through the switching 
and MUX box 265, for example, may be passed directly to an 
analog TV input. Also, data 95 entering through ports 130, 

35 135, 140, and/or 145 first passes through the UART 
(Universal Asynchronous Receiver and Transmitter) 160 in the 
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IO-ASIC (Input Output Application Specific Integrated 
Circuit ) 165. 

Analog data 85 to be sent to the remote control 10 
passes through "tuner and analog to digital converter" 125, 
which converts the data to digital data and tunes to a 
particular channel for transmitting to the remote control 10 
based on commands sent to the base station 75 from the 
remote control 10. The data 85 may then be processed by CPU 
155, such as perhaps to strip out only embedded data to pass 
along to the remote control 10, or to create an alternate 
view of a virtual world for a video game application. 
Converted data 85 is sent to an MPEG A/V encoder 170, 
whether or not the CPU performed any processing upon it. 

Digital data 85 and/or 95, or portions thereof to be 
sent to the remote control 10 may or may not pass through 
the MPEG A/V encoder 170, since data such as text will not 
necessarily require the MPEG compressing as video data does. 

Data 85 and/or 95 or portions thereof, whether 
originating from analog or digital data streams, then pass 
through MUX 175, either directly or after passing through 
MPEG A/V encoder 170 as stated. All data entering MUX 175 
at that point is digital data. MUX 175 combines MPEG A/V 
data streams with other data streams such as HTML data, then 
sends the combined data stream to an RF encoder 180 which 
passes the data through an antenna 185 so the data may be 
transmitted to the remote control 10. The HTML data sent to 
MUX 175 may be only part of. the HTML data stream 85 and/or 
95 originally entering the base station. 75, as CPU 155 may 
first process data 85 and/or 95 and determine, based on 
requests from the remote control 10, that some of it should 
be passed to TV 80 instead. 

Data 85 and/or 95 or portions thereof to be sent to an 
analog TV input are treated differently. Analog data is 
sent through the switching and MUX box 265 to be passed to 
the analog TV input, or to the analog to digital converter 
125 if necessary. Switching and MUX box 265 determines 
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which of multiple possible input signals to send to the 
input or inputs of an analog TV, based on commands from the 
remote control 10. If the analog TV has as many inputs as 
there are analog signals, and if the base station 75 has 
5 enough outputs connecting thereto, then the switching and 
MUX box 2 65 may send each of the analog signals to the 
analog TV. Before being sent to the analog TV, the analog 
data may undergo further processing as herein described, in 
which case the data typically would have to have first been 

10 converted to digital data. 

Digital data 85 and/or 95 or portions thereof to be 
sent to an analog TV input may also be processed by CPU 155, 
but that is not required. And before being sent to 
switching and MUX box 2 65 for transmission to the analog TV 

15 input, the data 85 and/or 95 or portions thereof must first 
undergo NTSC or PAL conversion, as indicated by NTSC/PAL 
encoder 27 0 . 

Data 85 and/or 95 or portions thereof to be sent to a 
digital TV input are also treated differently. Digital data 

20 85 and/or 95 enters through USB port 130 . or other digital 
port, and may be processed by CPU 155 just as the other data 
streams. Analog data is processed if necessary, after 
passing through the "tuner + analog to digital converter" 
125. The data and/or portions thereof are then transmitted 

25 to a digital TV input back through USB port 130 or other 
digital port. 

Turning to other specific hardware components, the data 
bus 150 also connects an RTC (real-time clock) 190, memory 
chips 195, 200, 205, and 210, the IO-ASIC 165, and port 130, 

30 which is a USB port. The other communications ports (135, 
140, 145, and control IR port 215), are connected to the IO- 
ASIC 165 as described herein. The base station 75 also has 
an alphanumeric display 220 for displaying status messages 
and the like. Power is supplied by an external power source 

35 225. 



WO 99/34599 



PCT/US98/2741S 



26 



The RTC 150 is preferably a crystal-based clock, and is 
used to synchronize IR commands with the proper TV 
broadcast, radio broadcast, satellite broadcast, or other 
signals to satisfy the consumer's requests. In addition, 
the RTC 150 may be used to determine when a particular title 
or titles from a CD have completed based on the play-time 
associated with the title or titles. 

Memory 195 is a boot ROM for holding the system 
software, and memory 210 is a flash ROM for storing 
downloaded information such as IR command libraries of 
devices being controlled, or other information such as data 
related to the user's CD collection. The flash ROM 210 is 
preferably at least 2-8 Megabytes. Memory chips 200 and 205 
are RAM chips used for working memory. Memory 200 is shown 
as a DRAM, and is preferably at least 8 megabytes. Memory 
205 is shown as an SRAM and is preferably at least 256-512 
Kilobytes. The SRAM 620 may be used for certain operations 
such as MPEG encoding and/or decoding that require faster 
speed than other operations. 

The IO-ASIC 165 comprises UART 160, a PWM (Pulse Width 
Modulator) 230, a 32-channel ICO (Interrupt Controller Unit) 
235, a 2-bank DRAMC (DRAM controller) 240, an LCD controller 
245, and its own 16-bit 4-channel timer 250. The UART 160 
is used to receive data through control IR port 215, and to 
transmit and receive data through ports 135, 140, and 145. 
Control IR port 215 or its equivalent is essential because 
the data received through control IR port 215 are the .IR 
commands from the remote control 10. The other ports (130, 
135, 140, and 145) are optional, but at least one is 
required in order for the base station 75 to connect to an 
outside data source such as the internet. Port 135 is an 
IrDA (Infrared Data Association) port used for wireless 
communication. Port 140 is an RS232 port for connection to 
a standard serial port of a PC. Port 145 is a modem port 
for- connection directly to telephone lines. Port 130 is .a 
USB port which is an industry standard port for "plug and 



WO 99/34599 PCT/US98/27415 

27 

play" connection to PCs. The USB port 130 is connected 
directly to bus 150 and can transfer data at a rate of up to 
4 megabytes per second, much faster than the UART 160 which 
can typically transfer data at a rate of only up to about 
5 115 kilobytes per second. The ports other than the USB port 
130 are connected to bus 150 only through the UART 160 in 
the IO-ASIC 165. 

The PWM 230 may be used to activate a speaker (not 
shown) to alert the user of errors on input when 

10 programming, or attempted unauthorized access to certain 
data, or a pending request, or any other event through the 
use of various beeping or buzzing sounds - of various 
frequencies, volumes, and/or durations. Voice data may also 
be employed where the voice data is either predetermined, or 

15 dynamically loaded into the base station 75, or dynamically 
created by suitable software. The ICU 235 coordinates and 
controls requests to the CPU 155 from various input and 
output devices either integrated with the base station 75 or 
connected thereto. The DRAMC 240 controls access to the 

20 DRAM 200, and LCD controller 24 5 controls what is displayed 
on the alphanumeric display 220. The timer 250 is used for 
short timing (on the order of milliseconds) within the IO- 
ASIC 165 (such as refreshing the alphanumeric display 220, 
etc.). 

25 The hardware of a typical remote control 10 will now be 

described, as seen in FIG. 4. The remote control 10 is 
capable of receiving RF data 275 through an RF antenna 280. 
The data 275 is transmitted from the base station's antenna 
185 (FIG. 3) , and, may include audio, video, and or other 

30 data such as HTML data. The data 275 passes first through a 
"transport Demux and MPEG A/V decoder" (Demux) 285 where any 
audio data -290, video data 300, and HTML or other data 305 
is separated. 

A data bus 310 connects, an RTC (real-time clock) 315, 
35 CPU .320,. memory chips. 325, 330, 335, and 340, IO-ASIC 345,- 
and USB. port 350. Various other communications ports 355, 
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360, 365, and 370, are connected to the IO-ASIC 345 as will 
be described shortly. The remote control has a touch screen 
375 and LCD 380, and may also have a speaker 385 for sound. 
The power source 390 is shown preferably as a 9V battery or 
9V from an external power source. 

The RTC 315 is preferably a crystal-based clock, and is 
used to synchronize IR commands with the proper TV 
broadcast, radio broadcast, satellite broadcast, or other 
signals to satisfy the consumer's requests. In addition, 
the RTC 315 is used to determine when a particular title or 
titles from a CD have completed based on the play-time 
associated with the title or titles. 

Memory 325 is a boot ROM for holding the system 
software. Memory 340 is a flash ROM, and stores downloaded 
information such as TV schedules , CD track data, pre-loaded 
IR command libraries, etc. The flash ROM 34 0 is preferably 
at least 2-8 Megabytes. Memory chips 330 and 335 are RAM 
chips used for working memory. Memory 330 is shown as a 
DRAM, and is preferably at least 8 megabytes. Memory 335 is 
20 shown as an SRAM and is preferably at least 256-512 
Kilobytes. The SRAM 335 is used for certain operations . such 
as video encoding and/or decoding that may require faster 
speed than other operations. 

The IO-ASIC 345 comprises a UART (Universal 
Asynchronous Transmitter and Receiver) 395, a 32-channel ICU 
(Interrupt Controller Unit) 400, a 2-bank DRAMC (DRAM 
controller) 405, an 8-channel CSC (Chip Select Controller) 
410, an LCD controller 415, its own 16-bit 4-channel timer 
420, and a PWM (Pulse Width Modulator) 425. The UART 395 is 
used to transmit data through port 355 which is a control IR 
port, and to transmit and receive data through ports 360, 
365, and 370. Control IR port 355 or its equivalent is 
essential because the data sent through control IR port 355 
are the IR commands to control devices. The other ports 
(320, 360, 365, and 370) are optional, but at least one is 
required in order for the remote control 10 to connect to an 
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outside data source such as the internet. Port 360 is an 
IrDA port used for wireless communication. Port 365 is an 
RS232 port for connection to a standard serial port of a PC. 
Port 370 is a modem port for connection directly to 
5 telephone lines. Port 350 is a USB port which is an 
industry standard port for "plug and play" connection to 
PCs. The USB port 350 is connected directly to bus 310 and 
can transfer data at a rate of up to 4 megabytes per second, 
much faster than the UART 395 which can typically transfer 

10 data at a rate of only up to about 115 kilobytes per second. 
The ports other than the USB port 350 are connected to bus 
310 only through the UART 395 in the IO-ASIC 345. 

The ICU 400 coordinates and controls requests to the 
CPU 320 from various input and output devices either 

15 integrated with the remote control 10 or connected thereto. 
The DRAMC 405 controls access to the DRAM 330, and the CSC 
410 processes the consumer's touch screen 375 requests, 
which first pass through a 10-bit analog-to-digital 
converter 430. The CSC 410 also interacts with the flash 

20 ROM 340, the modem port 370, and other components, and 
serves a fairly general control function in the 10 
subsystem. The LCD controller 415 controls what ' is 
displayed on the LCD display 380. The timer 420 is used for 
short timing (on the order of milliseconds) within the 10- 

25 ASIC 345 (such as refreshing the LCD display 380) . The PWM 
425 sends digital signals to a digital to analog amplifier 
435 which passes the signal to speaker 385. The speaker 385 
may be used to alert the consumer of errors on input when 
programming, or attempted unauthorized access to certain 

30 data, or a pending request, or any other event through the 
use of various beeping or buzzing sounds of various 
frequencies, volumes, and/or durations. Or the speaker 385 
may be used for voice sound as previously described. 

While certain embodiments are illustrated in the 

35 drawings and have just been described herein, it will be 
apparent to those skilled in the art that many modifications 
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can be made to the embodiments without departing from the 
inventive concepts described. Accordingly, the invention is 
not to be restricted except by the claims which follow. 
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Claims 

1 . A method of viewing multiple data streams 
substantially simultaneously comprising the steps of: 

providing a base station; 
5 providing a first device having a display, the 

first device being capable of displaying full motion video; 

providing a second device having a display, the 

second device being capable of displaying full motion videos- 
receiving a first data stream into the base 

10 stations- 
receiving a second data stream into the base 

stations- 
transmitting the first data stream from the base 
station to the first device; 
15 displaying the first data stream on the display of 

the first device; 

transmitting the second data stream from the base 
station to the second device; and 

displaying the second data stream on the display 
20 of the second device. 

2. The method of claim 1 wherein the first data 
stream comprises a broadcast TV signal. 

3. The method of claim 1 wherein the first data 
stream comprises a satellite TV signal. 

25 4. The method of claim 1 wherein the first data 

stream comprises HTML data. 

5. The method of claim 4 wherein the HTML data enters 
the base station from the internet. 

6. The method of claim 1 wherein. the first device is 

30 a TV. 
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7. The method of claim 6 wherein the second device is 
a hand-held remote control. 

8. The method of claim 7 wherein the second data 
stream is transmitted from the base station to the hand-held 

5 remote control using radio frequency transmission. 

9. The method of claim 1 further comprising the steps 

of: 

discontinuing to display the first data stream on 
the display of the first device; 

displaying the second data stream on the display 
of the first device; 

discontinuing to display the second data stream on 
the display of the second device; and 

displaying the first data stream on the display of 
15 the second device. 

10. A method of viewing different portions of data 
from a data stream on two physically separate displays 
substantially simultaneously comprising the steps of: 

providing a base stations- 
receiving the data stream into the base stations- 
processing the data stream within the base station 
to separate the data stream into a first portion and a 
second portions- 
transmitting the first portion from the base 
25 station to the first displays- 
displaying the first portion on the first displays- 
transmitting the second portion from the base 
station to the second display; and 

displaying the second portion on the second 

30 display. 

11. The method of claim 10 wherein the data stream 
comprises an analog signal having a vertical blanking 



20 
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interval with data embedded within the vertical blanking 
interval, and wherein the second portion comprises data in 
the vertical blanking interval. 

12. The method of claim 11 wherein the data stream 
5 comprises a broadcast TV signal. 

13. The method of claim 11 wherein the data stream 
comprises a satellite transmission signal. 

14. The method of claim 11 wherein the second portion 
comprises HTML data. 

10 15. The method of claim 10 wherein the data stream 

comprises a digital signal. 

16. The method of claim 10 wherein the second display 
is on a hand-held remote control. 

17. The method of claim 10 wherein the first display 
15 is on an internet-enabled TV. 

18. The method of claim 10 further comprising the 
steps of: 

discontinuing to display the first portion on the 
first display; 

20 displaying the second portion on the first 

display; 

discontinuing to display the second portion on the 
second display; and 

displaying the first portion on the second 

25 display. 

19. A method of viewing multiple channels of a TV 
broadcast signal substantially simultaneously comprising the 
steps of: 
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display; 



station; 



providing a base station having a tuner; 
providing a first device having a tuner and 



a 



providing a second device having a display; 
receiving the TV broadcast signal into the base 



sending the TV broadcast signal from the base 
station to the first device; 

tuning the first device to a predetermined channel 
using the tuner of the first device; 

displaying the channel of the TV broadcast signal 
tuned to by the tuner of the first device on the display of 
the first device; 

tuning the base station to a predetermined channel 

using the tuner of the base stations- 
sending the channel of the TV broadcast signal 

tuned to by the tuner of the base station from the base 

station to the second device; and 

displaying the channel of the TV broadcast signal 

sent to the second device on the display of the second 

device . 

20. The method of claim 19 wherein the second device 
is a portable hand-held device. 

21. The method of claim 19 wherein the channel of the 
TV broadcast signal sent from the base station to the second 
device is sent using radio frequency transmission. 

22. A system for providing enhanced viewing of 
multiple data streams comprising: 

a first device having a display, the first device 
being capable of displaying full motion video; 

a second device having a display, the second 
device being capable of displaying full motion video, and 
having communications hardware for sending control infrared 
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(IR) commands, and a radio frequency <RF) antenna for 
receiving RF transmissions; and 

a base station connected to the first device by a 
cable, and having communications hardware for receiving 
5 control IR commands and for sending RF transmissions. 

23. The system of claim 22 wherein the second device 
is a hand-held remote control. 

24. The system of claim 23 wherein the remote control 
further comprises a USB port. 

10 25. The system of claim 22 wherein the first device 

has communications hardware for receiving control IR 
commands . 

26. The system of claim 22 wherein the base station 
further comprises a tuner. 

15 27. The system of claim 23 wherein the remote control 

comprises a tuner. 

28. A method of swapping channels from a display on a 
first device to a display on a second device, and from the 
display on the second device to the . display on the first 

20 device, wherein the channels are both at least initially 
contained within a single data stream, and wherein the first 
device has communications hardware for transmitting infrared 
( IR) commands and the first device is receiving the channel 
on its display via RF communications from a base station, 

25 and the . second device has communications hardware for 
receiving IR commands, comprising the steps: 

actuating a button on the first device, said 
actuation causing an infrared (IR) command sequence to be 
sent from the first device to the second device commanding 

30 the second device to adjust a tuner therein to a 
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predetermined channel, said actuation also causing another 
IR command sequence to be sent from the first device to the 
base station commanding the base station to adjust a tuner 
contained therein to a predetermined channel.. 

29. The method of claim 28 wherein the first device is 
a hand-held remote control, and the second device is a TV.. 

30. The method of claim 29 wherein the remote control 
has a memory and wherein the remote control stores in the 
memory information related to the IR commands sent such that 
the remote control is aware of the channels tuned to by the 
tuners . 

31. The method of claim 7 wherein the second data 
stream is transmitted from the base station to the remote 
control using Firewire 1394 protocol wired communication. 

32. The method of claim 15 wherein the digital signal 
conforms to a protocol wherein there are dedicated slots for 
video data, audio data, and other data, and wherein the 
second portion comprises the other data. 

33. The method of claim 32 wherein the protocol is 
MPEG and the other data is HTML data. 

34. The method of claim 16 wherein the second portion 
is transmitted from the base station to the second display 
via Firewire 1394 protocol wired communication. 

35. The system of claim 23 wherein the remote control 
further comprises a Firewire 1394 port. 

36. The method of claim 29 wherein the remote control 
sends information to the base station via RF communications. 
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1 

DESCRIPTION 

System For Dual-Display Interaction With Integrated 
Television And Internet Content 

Related Applications 
5 This application contains subject matter related to co- 

pending U.S. patent application filed under attorney docket 
number 228/269, entitled "Portable Internet-Enabled 
Controller and Information Browser For Consumer Devices," 
filed December 31 , 1997, invented by the same inventor as 
10 the present invention, and to be assigned to the same 
assignee as the present invention. The entire original 
disclosure of said co-pending application is herein 
expressly incorporated by reference. 

Terminology 

15 As used in this application, the term "data stream" 

refers generally to transmitted information, and a data 
stream may itself contain multiple data streams. A data 
stream may be HTML data transmitted from the internet, or it 
may be a "media stream" such as an analog or digital TV 

20 broadcast signal, satellite TV signal, cable TV signal," or 
other audio and/or video signal. Media streams such as TV 
broadcast signals may contain several channels, and each 
channel may further contain audio, video, or other embedded 
data streams, including HTML data. Furthermore, HTML data 

25 is used to refer to any internet-derived data, as opposed to 
solely data that is in the HTML protocol format, as the 
concepts described herein are equally applicable to other 
internet-derived data, especially data that is transmitted 
in a widely accepted protocol. 



30 



Field Of The Invention 

The present invention relates generally to systems and 
methods for using two or more cooperating but physically 
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independent displays for enhanced viewing of data streams on 
each display, wherein the viewing of a data stream on one 
display does not interfere with the viewing of a data stream 
on the other displays. More particularly, the images may be 
derived from complex data streams such as multiple TV 
(television) broadcast signals (and/or cable TV or satellite 
transmissions or other video signals), internet data, and/or 
other local data such as information related to the state of 
a consumer's home appliances. In addition, the different 
displays may be available for viewing by more than one 
person. 

Background Of The Invention 

As technology rapidly advances, there are more and more 
sources and types of information available to a consumer. 
For example, after TVs improved from simple black-and-white 
TVs to color TVs, cable and satellite TV became available. 
Most recently, internet-enabled TV became available to 
provide both TV and internet content. In a fast-paced 
society such as ours, consumers are looking for ways to 
enhance the overall viewing experience and to absorb more of 
the available information from various sources, in less 
time . 

One of the simplest methods of obtaining more 
information is to use multiple independent displays, such as 
a having two TVs tuned to two separate channels, or two 
closed circuit cameras (e.g., security cameras) focussed on 
separate areas. The problem with this method is that the 
person viewing the information may have to use various non- 
cooperating controllers for controlling the different 
devices, and may even have to physically change locations 
after viewing one display in order to be able to view 
another display. Additionally, this type of method usually 
involves fixed displays, as opposed to at least one portable 
display. And though it is possible to use a single 
universal remote control to control both TVs, additional 
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problems exist including the cost of multiple TVs or 
monitors, the physical space required for the TVs, and the 
availability of other electronics hardware such as cable 
receivers, closed circuit wiring, etc. 
5 Other multiple-display systems have displays in close 

physical proximity to each other and may display related 
data, but if integration of the data is to occur, it must be 
accomplished manually, as the underlying systems are not 
capable of doing so. That is because the physical display 

10 devices are each dedicated to providing a limited amount of 
information related to a limited topic of interest. 
Examples of these types of systems include radar screens and 
controls in a cockpit of an airplane, and medical 
instruments monitoring various vital signs of a patient in a 

15 hospital. 

One method of enhancing the viewing experience involves 
using multiple displays to simultaneously display multiple 
duplicate data streams. This is common in electronics 
stores where many TVs are all tuned to the same channel, or 

20 in exercise classes where many TVs are all playing the same 
exercise video tape. Though perception of the data stream 
(i.e., TV program or video tape) might be enhanced in such a 
case, the viewer is only viewing multiple copies of the same 
data stream, and is not benefitted with access to data 

25 streams of differing content. 

Other multiple-display systems display separate parts 
of a single image, and the displays taken as a whole are 
designed to represent the desired image. The displays are 
arranged in a predetermined pattern such as side-by-side, in 

30 a matrix, or even to form a cylindrical "circle vision" 
theater. Again, though the viewing experience is - altered 
and perhaps enhanced, the viewer is only viewing what is 
designed to be a single image from multiple data streams. 
In fact, these systems may even detract from the desired 

35 image perception due to the gaps between the several display 
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screens and/or the imperfect placement of the various parts 
of the overall image within the various display screens. 

Another attempt to provide consumers with a method of 
viewing multiple data streams simultaneously to obtain more 
information, is through the use of windows-type operating 
systems used with PCs. The user may swap between multiple 
windows to view various data streams. However, these 
systems are designed under the assumption that the viewer 
concentrates on a single type of application at a time. 
Thus, the same physical display is used for the multiple 
windows that represent various applications. A particular 
window may be enlarged to cover the entire display, but then 
the benefit is lost of having the capability to view 
multiple data streams substantially simultaneously. 
Furthermore, the individual applications are typically 
independent of each other, and do not communicate with each 
other except for perhaps during data transfer. Even where 
multiple windows are viewed simultaneously, such as during 
video conferencing where a person may view multiple 
conference "attendees" in various windows, the images of the 
attendees all appear on the same physical display, and the 
more attendees there are, the less room there is for each to 
be displayed. 

Various systems also exist that attempt to solve the 
problem of viewing multiple data streams substantially 
simultaneously with two or more cooperating displays. A 
common system is a PIP (picture-in-picture) system, which 
allows simultaneous viewing of two TV programs at once, for 
example. Though the viewer may "channel surf" a first data 
stream without affecting the overall image produced by the 
second data stream, both data streams are displayed on the 
same physical display, and therefore there is some 
degradation of the viewing experience of each. For example, 
a primary program may be playing on the entire physical TV 
display while a secondary program (the PIP) is playing in a 
small area in the corner of the display. The primary 
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program is degraded due to the screen real estate used up by 
the secondary program, and the secondary program is degraded 
because it is not able to occupy the full size of the 
physical display. 
5 Similarly, existing TV interfaces provide a wide 

variety of available functions such as data display (e.g., 
to adjust attributes of the TV and related devices), 
advertisements, internet browsing, alternate channel 
viewing, etc. But the additional functions are all managed 

10 by assigning different amounts of the same physical screen 
to the additional function or functions . This approach of 
sharing the same physical screen real estate for all 
operations performed by a device means that combined 
activities restrict the display potential of each other, 

15 unless each activity occurs serially, which would defeat the 
entire purpose of allowing simultaneous viewing of multiple 
data streams in the first place. Moreover, the resolution 
of standard TV display formats such as NTSC (the American 
standard) and PAL (the British standard) are not very 

20 suitable for displaying static text. 

Another type of system that involves data integration 
to some extent is a computer network, or even simply a 
remote PC communicating with a host. Software synchronizes 
the data files stored on each device, and files can then be 

25 downloaded from the host to the remote PC, edited remotely 
on the PC, and then transferred back to the host. However, 
this setup is designed primarily for one or the other device 
to be used independently. The simultaneous use of both 
devices occurs only during file transfer, or in some cases 

30 to achieve the same type of extended screen real estate for 
a single-image display that occurs with two or more displays 
side-by-side. 

Thus it would be desirable to provide systems and 
methods for using two or more cooperating but physically 
35 independent displays for achieving enhanced viewing and/or 
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browsing of data on each display, without affecting the 
viewing and/or browsing of data on the other displays. 

Summary Of The Invention 

The present invention is based upon the recognition 
that there are many situations in which users wish to view 
or otherwise perceive more than one data stream 
substantially simultaneously, without having the experience 
of any data stream degraded by the others. A hand-held, 
portable remote control with an integrated video display 
capable of displaying full motion video is thus disclosed, 
in combination with hardware and/or software that enables 
interaction between the TV or other primary display screen 
and the hand-held display. The hardware may include an 
integrated TV tuner and/or various data communication ports, 
or a physically separate base station with a TV tuner and/or 
various data communication ports. In the latter case, the 
base station and the remote control would communicate 
preferably via RF (radio frequency) . 

Many applications exist where the user or users may 
derive a great benefit from an additional display screen for 
user interaction, as will be discussed herein. One such 
situation is for users of TV systems that display text-based 
status information on the TV display during attribute 
adjustment (e.g., to volume, contrast, color, tint, 
brightness, sound, etc.), or for closed caption messages 
such as sports scores, stock quotes, etc. The remote 
control display may be used to display that information and 
therefore leave the primary viewing screen free of 
unnecessary clutter. For existing internet-enabled TV the 
benefit realized is even greater due to the significant 
amount of screen real estate typically devoted to displaying 
navigational and other information. By adding a second 
display that communicates with the TV, all of the 
navigational interface can be removed from the TV screen, 
leaving much more room for actual viewing content. 
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Video games (both single user and multi-user) are 
another application where the users may realize a benefit 
from an additional display. Many computer games exist that 
allow multiple players in different locations to play with 
5 (or against) each other. These often take the form of 
interaction in on-line "virtual worlds." In order to play 
these games effectively, a user must be aware of the general 
topology of the virtual world, as well as where he or she is 
in the world in relation to others. The interface to the 

10 virtual world may be presented in various perspectives, and 
from various distances. In any case, the user's sense of 
where he or she is in the world can be greatly enhanced by 
the provision of an additional view. 

Similar advantages apply to other types of virtual 

15 world interactions, such as three-dimensional multi-user 
chat rooms, where the present invention would enable 
simultaneous first-person and alternative more global views 
from selected vantage points. This will greatly assist the 
navigation through the virtual world, and also increase the 

20 perceived- immersion in the world by avoiding constant 
context-switching on a single screen. 

Another application of the present invention finds use 
in working with internet-enabled TV. Several companies are 
working on (or are marketing) technology that integrates TV 

25 and internet content within the same signal. For analog 
signals such as TV broadcasts, internet data is embedded 
within the VBI (vertical blanking interval) . Examples of 
this are Intel 1 s "Intercast" standard (which some TV 
networks are already using) , and Navio's HTML-TV standard. 

30 Likewise, Thomson Sun Interactive has been using an "Open 
TV" standard in Europe, in which the technology for both 
broadcasting and receiving signals with data embedded using 
the Open TV standard has been implemented. Another company, 
Starsight, markets hardware and "software capable of 

35 receiving and decoding signals with embedded data, where the 
hardware and software is for use with third party units. 
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Data may also be embedded in digital signals using the MPEG 
standard. The embedded data typically may be displayed as 
text and graphics independently of the image displayed from 
the normal video signal. 

The present invention would allow both the regular 
signal, and the embedded data, to be viewed simultaneously 
on different displays - something that is not possible using 
the technology provided by the aforementioned systems. This 
not only provides clearer viewing of both data streams due 
to elimination of screen clutter, but also allows "semi- 
synchronous" interactions with the TV and internet data, as 
described herein. 

The present invention is especially useful in family or 
group situations where multiple people are viewing the same 
TV screen simultaneously. One person could channel surf 
actual TV content on the hand-held remote control, while the 
other viewers enjoy the TV program on the primary display 
(e.g., the TV screen). This effectively turns a legacy TV 
into a TV with PIP capability, with the added benefit that 
the PIP is actually on a separate display. Using suitable 
communications infrastructure, which is described in more 
detail herein, the remote control's display may also 
function as a regular portable TV. 

Additional uses of the present invention may be for 
video-conferencing, wherein each of many displays may be 
used to show one attendee of the conference, or for 
controlling other various consumer devices and appliances 
such as those that respond to IR (infrared) commands, as 
disclosed in previously-referenced co-pending application 
[attorney docket #228269]. The . full motion video capability 
of the remote control allows the user to view the output 
from security cameras, a video baby monitor, etc., or to use 
the remote control and its display to form part of a video 
phone system (with the addition of microphones), all without 
interrupting the primary display screen such as the TV 
display. And in an embodiment where the remote control has 
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enough memory or access to an outside storage device, the 
remote control may also function as a convenient front-end 
to DVD (digital versatile disk) players and recorders/ 
allowing the user to browse actual video, audio, and other 
5 recordings to select a desired title for playing. 

Accordingly, it is an object of the present . invention 
to provide improved systems and methods for providing an 
enhanced overall viewing experience of multiple data 
streams . 

10 It is a further object to provide improved systems and 

methods for allowing substantially simultaneous viewing of 
multiple data streams. 

It is a further object . to allow substantially 
simultaneous viewing of multiple data streams using two or 
15 more cooperating but physically independent displays. 

It is a further object to provide a hand-held remote 
control with an integrated video display for providing the 
means by which one of the data streams is viewed. 

Other objects and advantages . of the present invention 
20 will become apparent from the detailed description of 
preferred embodiments which follow, when read in conjunction 
with the drawings. 

Brief Description Of The Drawings 

FIG. 1 shows a preferred embodiment of the remote 
25 control of the present invention with a variety of physical 

actuating buttons, and a blank display. 

FIG. 2 is a system diagram of a preferred embodiment of 

the present invention, showing the channels of communication 

between the various system components. 
30 FIG. 3 is a high-level schematic of hardware used in a 

preferred embodiment of a base station of the present 

invention that is used in conjunction with a remote control 

of the present invention. 
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FIG. 4 is a high-level schematic of hardware used in a 
preferred embodiment of a . remote control of the present 
invention. 

Detailed Description 

The present invention will be described as a "dual 
display" system, but it is to be understood that the 
concepts described herein are equally applicable to a 
"multiple display" system having three or more cooperating 
displays. Of course, implementation of the latter system 
would require more hardware and software for managing the 
communications between the various displays. 

Before describing the operation of the system and 
methods of the present invention, it will be helpful to 
describe the physical appearance of a preferred embodiment 
15 of a remote control 10 that forms part of the system. 
Turning to FIG. 1, the drawing is not to scale, but 
preferably the remote control 10 is a hand-held size on the 
order of only several inches long by several inches wide by 
a few inches deep, such that it may be carried comfortably 
in one hand or in a shirt pocket. 

The display area 15 is surrounded by various physical 
actuating buttons 20, 25, 30, 35, 40, 45, 50, 55, and 60. 
There may be, for example, up to twenty physical buttons, or 
less than twenty, and more preferably less than fifteen, and 
25 more preferably less than twelve, and still more preferably 
ten or less. The display area 15 preferably comprises an 
LCD and is preferably 320 pixels by 240 pixels (H VGA) . It 
may have higher resolution (i.e., greater than 320 pixels 
and/or greater than 240 pixels) or lower resolution, but 
30 preferably not lower. It may be monochrome or color, and is 
preferably touch sensitive. it must be capable of 

displaying full motion video, such as the images produced by 
broadcast TV signals. 

The physical actuating buttons may include push buttons 
35 (e.g., 20, 30, 35, 40, 45, and 50), switches (e.g., 25), 
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sliders (e.g., 55 and 60 which slide in slots 65 and 70 
respectively) , or other mechanisms not shown such as touch- 
sensitive pads, levers, toggles, dials, knobs, pull buttons, 
track balls, a mouse, etc. The functions of the buttons may 
5 vary, and some may have fixed functions while others may be 
programmable. Common functions may include a power on/off, 
a help button, and a mode-switching, button wherein * the 
remote control 10 is switched from a viewing mode to a 
control mode. The display 15 may also have some 

10 programmable buttons (also known as "soft keys) " in certain 
modes of operation. A more detailed explanation of the 
possible operation and function of the buttons is set forth 
in previously-referenced co-pending U.S. patent application 
[attorney docket #228/269] . In. addition, further uses may 

15 be made of the video capability of the remote control 10 
here, such as by using video clips during configuration 
and/or during "help" mode. 

The remote control 10 is used in combination with 
hardware and/or software to form a system that enables 

20 interaction between the TV or other primary display, and the 
display 15 of the remote control 10. The applications of 
the system as previously discussed are described here in 
slightly more detail, and a description of the hardware 
and/or software that enables the system to be used as herein 

25 described will follow. 

For the situation where an internet-enabled TV display 
is cluttered with text-based and other navigational 
information, a further benefit of the present invention is 
that with suitable HTML (Hypertext Markup Language) parsing 

30 software 10, some parts of the actual internet content could 
also be displayed on the remote control's display 15. This 
frees up even more of the primary display for actual 
content. For example, many web sites employ "frames" with 
their own navigational aides in the form of graphical icons, 

35 indexes to sub-pages, etc. These could be displayed on the 
hand-held remote control, again leaving more room for the 
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image created by the primary data stream to be shown on the 
TV screen. Some of the buttons on the remote control 10 in 
this context could be used for bookmarking web sites, 
context switching (e.g., from browsing the web to watching a 
TV program), internet navigation, etc. 

Moreover, for both internet-enabled TV and regular TV, 
an additional benefit to be realized is that the resolution 
of the text-based information on the remote control display 
15 could be much greater than that on the TV displays 
because the remote control display 15 may have an LCD 
(liquid crystal display) as opposed to a CRT display, and 
therefore may employ VGA, SVGA or better technology as 
opposed to NTSC or PAL. The function of one of the buttons 
on the remote control 10 in this application may be to swap 
the images shown on the displays. Other functions for which 
sliders 55 and 60 are particularly suited could be used to 
change channels on the TV and on the remote control's 
display 15 respectively. 

During virtual world communications, such as in virtual 
world games (single or multiple user), the benefit of the 
present invention is that the interface to the virtual world 
may be presented in various formats on the two separate 
displays. The player is thus provided with more strategic 
information. Various perspectives could be a first person 
view, wherein the virtual camera takes the viewpoint of a 
character in the world that is being played by the user; a 
second person view, wherein the virtual camera takes the 
viewpoint of a character in the world that is being played 
by a different user; or a third person view, wherein the 
virtual camera takes the viewpoint of no particular 
character, but instead has a global view of the world, 
usually focussing on the character being played by the user. 
The two simultaneous views may be from different 
perspectives, or they may be from the same perspective, with 
one view displaying a close-up while the other displays a 
"bird's eye" view, or any other combination. . Here too, the 



WO 99/34599 PCT/US98/274I5 

13 

function of one of the buttons on the remote control 10 may 
be to swap the images shown on the. displays. Or one 
function may be to enlarge or reduce a particular display, 
or to cycle through the available displays so that a player 
5 may perceive the world as one of his or her "enemies" or 
"allies" is perceiving the world. Other functions may be 
for other game controls, which may also be performed via an 
•IR keyboard or other, input means. 

When using the present invention with internet-enabled 

10 TV, an additional perceived benefit is in the use of "semi- 
synchronous" interaction with TV ads. TV ads on internet- 
enabled TV typically provide addresses or links to web 
sites, where a viewer could obtain more detailed information 
and review the information at their own pace. With the 

15 present invention, a viewer could use the remote control 10 
to bookmark web sites from advertisements as they occur or 
soon thereafter, and then later use the remote control 10 to 
contact the web sites for further information. The viewer 
could even do so during the time when regular programming 

20 .(non-ads) returned to the TV, and browse the internet sites 
on the remote control's display 15 without affecting or 
interfering with the primary TV display. Furthermore, 
existing internet-enabled TV systems typically require the 
use of navigational input means (such as a mouse or IR 

25 keyboard) to access specified functions or to enable 
internet links appearing on the display. With the present 
invention, the same type of navigational aides could be used 
for accessing information on the remote control's display 
15. However, it is possible, and desirable, for such access 

30 to be accomplished directly through the use of buttons, thus 
preventing the need for excess input and navigation. 

The present invention also, provides a benefit for the 
situation where additional data (e.g., HTML data) is 
embedded in the VBI of the TV signal, or in dedicated slots 

35 within a digital signal. That data could be stripped from 
the signal . and viewed on the remote control's display 15 



WO 99/34599 



PCT/US98/27415 



while the regular video signal appeared on the primary 
display. This differs from how current internet-enabled TVs 
make use of VBI data, because they all operate under the 
presumption that only one display is available for both the 
5 regular video signal and the VBI data. Some of the buttons 
on the remote control 10 in this context could be used for 
changing font sizes of any text data, scrolling through 
text, or swapping the programs showed on the two displays. 

An especially useful application of the present 

10 invention is in the context of family or group TV viewing is 
the ability to channel surf. This may also be useful for a 
single viewer. A viewer may channel surf for alternate 
programs of interest, without interfering with the program 
then playing on the TV. The advantages of being able to see 

15 alternative channel program listings on a hand-held display 
separate from the main viewing screen are described in the 
previously referenced co-pending application [attorney 
docket #228/269], The present invention also enables the 
user to actually select the channel, and view the alternate 

20 program on the remote controls display 15. Headphones may 
be used to receive the audio portion of the alternate 
channels being surfed. After finding a desired alternative, 
a user may execute a command (by the press of a button for 
example) to cause the primary viewing screen to display the 

25 desired program that was found while surfing. In addition, 
a button may allow the user to execute a "swap" such that 
not only does the primary display switch to show the desired 
program that was found while surfing, but the hand-held 
display 15 switches to show the program that was being 

30 played on the primary display as well. 

Turning now to the hardware and/or software of the 
present invention, it is to be understood that the drawings 
are of preferred embodiments only, and that other 
configurations may be designed without varying from the 

35 spirit and scope of the inventive concepts described herein. 
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The software may be written in any convenient 
programming language or languages, and preferably comprises 
an underlying system control component and an application 
interface component. The system component controls the 
5 execution of the actual commands necessary to control the 
consumer devices. It controls the storage, access, and 
other data management tasks necessary for proper 
functioning. It also retains an internal representation of 
each device being controlled, its type (CD player, TV, 

10 etc.), name, and associated icon if applicable. The system 
component also maintains a table of IR commands for each 
device, along with associated data such as the 
representation of the commands on the remote control's 
display, if applicable. The application interface component 

15 acts as a front-end for the consumer to program and 
otherwise make use of the system component, and is 
implemented such that a control algorithm allows the 
consumer to navigate through the available screens to 
accomplish the desired control over consumer devices. 

20 The software is preferably programmable by the user and 

presents a unified system view of various devices being 
controlled. For example, there may be volume and channel 
controls on one screen which control the volume of a stereo 
speaker connected to a TV output and the channel on a cable 

25 box connected to the TV input respectively. The ease of 
this system contrasts with that of known universal remote 
controls where it is necessary to first select a "stereo" 
mode, then change volume, then select "cable" mode to change 
channel . 

30 The programming may be accomplished with an integrated 

graphical keyboard for simple tasks such as entering and 
modifying passwords, or for more complex programming, but 
the latter is preferably done with the use of companion 
software either using an IR keyboard or other input means in 

35 conjunction with the display 15, or by using PC software on 
a PC. For example, the functions to be performed by certain 
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buttons may be programmed by companion software, and the 
display may then show the association of physical or touch 
screen actuating buttons with the actual functions they 
perform. Passwords and filters may be programmed to assist 
parental control or other monitoring, and screen layouts may 
also be designed. For a more detailed description of 
associated software, the previously mentioned co-pending 
U.S. patent application [attorney docket # 228269] may be 
consulted. 

Turning now to the hardware, FIG. 2 shows a system 
diagram of how components of the present invention cooperate 
with existing technology to implement the concepts described 
herein. The hardware for the base station 75 will be 
described in more detail in conjunction with the description 
of FIG. 3, and the hardware for the remote control 10 will 
be described in more detail in conjunction with a 
description of FIG. 4. 

FIG. 2 shows the base station 75 as a physically 
separate component from the remote control 10, which is a 
preferred embodiment, but the base station 75 may be 
implemented as an integrated part of the remote control 10 
or of the TV 80 if desired. In fact, some existing ATVs 
(advanced TVs) are equipped with a two-way internet 
connection, a decoder for separating out embedded data from 
analog or digital signals, and hardware and software to send 
complex data streams including video signals. For users 
with ATVs such as this, the remote control 10 needs only to 
have the capability of receiving and displaying complex data 
streams, and needs no additional communication ports other 
than a control IR port for sending IR commands. In such an 
embodiment, the elimination of additional communication 
ports and/or a decoder may have a significant effect on the 
cost of the remote control 10. 

Other advantages of having a base station 75 separate 
from the remote control 10 (i.e. r either physically 
independent or integrated in the TV 80) are that the remote 
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control 10 may be lighter in weight and smaller in size. 
And most significant, the remote control 10 could operate in 
a wireless mode, with the base station 75 having any needed 
wired connections to the home cable box, satellite TV 
5 receiver, VCR, etc. Size, weight, and the ability to 
operate in a wireless mode are critical features in 
providing an overall enhanced viewing experience. 

Still referring to FIG. 2, various signals (data 
streams) 85 from various sources enter the base station 75 

10 as indicated by arrow 90. The signals 85 may originate from 
broadcast TV, cable TV, satellite TV, a VCR, a laser disc, a 
DVD, or any other source, they may be analog or digital, and 
they may contain embedded data as previously described. 
Other data 95 from an outside data source such as the 

15 internet may enter the base station 75 as indicated by arrow 
100. In practice, it is preferable, as indicated by FIG. 2, 
that all data streams enter the base station 75, which may 
then act as a switching station for the data streams by 
determining whether to send the data streams (or parts 

20 thereof) to the TV 80, to the remote control 10, to both, or 
to neither. This would be extra beneficial, for example, if 
the TV 80 had only one input, because instead of having to 
physically switch cables to change from one input to 
another, the switching could be accomplished within the base 

25 station 75 and the base station 75 could send the proper 
input signal to the TV 80. However, it is technically 
feasible to have incoming signals sent both to the base 
station 75 and to the TV 80. 

Data streams pass from the base station 75 to the TV 80 

30 by wired connections as indicated by arrow 105. There may 
be only one connection, or there may be many. A typical 
connection may include a standard TV cable which passes all 
or part of data streams 85 to the TV 80. Other connection 
means (such as RS232, USB, and/or modem ports or other 

35 digital I/O ports for DVD media, etc.) may exist in the TV 
80 for passing all or part of data streams 95 to the TV. 
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The latter connections would apply, for example, to ATVs 
with internet capability. 

Data streams pass from the base station 75 to the 
remote control 10 by wired or wireless communications, 
5 indicated by arrow 110. Wired communications may employ 
either USB or Firewire 1394 protocols, both of which are 
capable of transmitting full motion video. However, wired 
communications are not preferred, as they detract from the 
overall viewing experience by inhibiting the user's 
10 mobility. 

For wireless communications, the base station 75 may 
transmit data to the remote control 10 by way of fast IrDA 
or RF, but the preferred method is RF, in which case the 
frequency of transmission would be preferably at 900MHz, 
15 2.4GHz, or other FCC-approved home communications 
frequencies. The main reason RF is preferred is that an 
important aspect of the present invention is the ability to 
view video on the remote control's display 15. For the base 
station 75 to efficiently and accurately send video signals 
at an acceptable rate of speed, IR is not as well-suited as 
RF. IR transmissions have limited range capabilities, and 
are useful typically only for sending small amounts of data, 
not the large amounts of data in a video signal. Even "fast 
IrDA" technology, which is capable of transmitting video 
25 signals, can operate effectively only within about one meter 
at most. Furthermore, IR signals will not operate through 
walls, whereas RF signals will, thus with RF communications 
a user may operate the remote control 10 from a room 
physically separate from the room where the base station 75 
30 is located. 

Communications from the remote control 10 to the TV 80 
(as indicated by arrow 120) are preferably by standard 
control IR. Typical communications from the remote control 
10 to the TV 80 may be to adjust TV attributes such as 
35 volume, channel, input, etc. The remote control 10 may also 
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send commands to other devices, such as a DVD player, VCR, 
etc., for the control thereof. 

Communications from the remote control 10 to the base 
station 75 (as indicated by arrow 115) are also preferably 
5 by standard control IR, though RF communications to the base 
station 75 could be an option because some of the hardware 
would already be present .due to the need for the base 
station 75 to transmit by RF to the remote control 10, as 
previously described. Typical commands from the remote 

10 control 10 to the base station 75 may be a request for a new 
channel to view on the display 15 (i.e., channel surfing), a 
request to swap displayed programs with the TV 80, a request 
to access new data from the internet or other outside data 
source, or any other request to the base station 75 for data 

15 streams. 

The type of request from the remote control 10 to the 
base station 75 will vary greatly depending on the 
application in progress. For example, during a video game, 
a request may be to provide another view of the virtual 

20 world topology. The base station 75 may satisfy that type 
of request by retrieving the data 95 from an outside source 
such as the internet, or alternatively the base station 75 
may do processing internally to provide the other view. 
Other types of requests may be to access a new web site 

25 while browsing the internet, in which case the base station 
75 would access the data 95 and transmit it to the remote 
control 10. 

An extremely useful feature is to have the data streams 
on the TV's display and on the remote control's display 15 

30 swapped. In order for this to occur, the base station 75 
that is providing a channel to the remote control 10 must 
tune to a new channel (the one the TV is tuned to just prior 
to the swap) , and the TV must tune to a new channel (the one 
the base station is tuned .to just prior to the swap) . 

35 Therefore,, there needs to be a way for the system (e.g., as 
seen in FIG 2) to know what channel the TV 80 is tuned to 
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just prior to the swap, and what channel the base station 75 
is tuned to just prior to the swap. A preferred method, 
though by no means the only method, is for both the base 
station 75 and the TV 80 to get the required information 
from the remote control 10. This will usually be possible 
because the remote control 10 is aware (i.e., may store in 
its memory) of the channel the TV is tuned to based on the 
fact that the remote control 10 will have sent the "change 
channel" IR command to the TV 80 in the first place. 
Similarly, the remote control 10 will be aware of the 
channel the base station 75 is tuned to based on the fact 
that the remote control 10 will have sent the "change 
channel" IR command to the base station 75 in the first 
place too. Thus, the remote control 10 may be programmed to 
send IR control commands to both the TV 80 and to the base 
station 75 with the push of a single button. 

So if the user wants to swap the programs being 
displayed on the TV 80 and the remote control's display 15, 
a single button on the remote control may be programmed to 
execute the swap. An example of the sequence of commands 
sent by actuating the swap button is as follows. First, a 
command is sent from the remote control 10 to the base 
station 75 to tune the base station's tuner to the channel 
at which the TV 80 is tuned. As previously stated, the 
remote control 10 already knows the channel. That would 
complete the first half of the swap, as the program then 
being displayed on the TV 80 would be displayed on the 
remote control's display 15 due to the base station 75 
tuning to the proper channel. Note that the remote control 
10 will have known the channel that the base station 75 was 
tuned to just prior to the swap as previously stated, and 
would have stored that information in its memory for 
subsequent use to complete the swap. That information is 
needed because next, a command is sent from the remote 
control 10 to the TV 80 to tune the TV tuner to that 
channel, i.e., the channel that the base station 75 was 
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tuned to just prior to the swap. The swap is now completed, 
as the TV would then be tuned to the channel that the base 
station 75 was tuned to just prior to the swap, and would 
therefore be displaying the program that was being displayed 
5 on the remote control's display just prior to the swap. The 
remote control 10, of course, would retain in its memory the 
channels which it commanded both the base station 75 and the 
TV 80 to tune to, so that the process could be repeated. 

Adding a slight twist to the previous example, assume 

10 the user is viewing channel 1 on the TV display, and channel 
2 on the remote control's display 15. In that situation, 
the TV 80 is tuned to channel 1. The base station's input 
signal 85 is from broadcast TV, and its tuner is set to 
channel 2. Now, the user wants to watch a video on the 

15 remote control's display 15. The user will send a command 
to the base station 75 to switch its input 85 from broadcast 
TV to the VCR, and a command to the VCR to begin playing. 
These commands could of course be sent by actuating a single 
button on the remote control 10. Now, if the user wants to 

20 execute a swap so that the TV 80 is displaying the video, 
and the remote control 10 is displaying channel 1, the 
following is a typical sequence of commands that would be 
sent. First, a command sequence is sent from the remote 
control 10 to the base station 75 for the base station 75 to 

25 switch its input to broadcast TV, and to tune its tuner to 
channel 1. The first half of the swap is complete. Next, a 
command is sent to the base station 75 to output from the 
switching and MUX box 265 the VCR signal, instead of the 
broadcast TV signal. Next, a command is sent to the TV to 

30 change to channel 3 (or whatever the appropriate channel is 
for VCR input) . The second half of the swap is now 
complete. The TV 80 would be showing the video, and the 
remote control 10 would be showing channel 1. Again, the 
remote control 10 could be programmed such that these 

35 commands could be sent by actuating a single button on the 
remote control- 10. . . 
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For applications where embedded data is to be separated 
from the signal 90 or other data 95, the remote control 10 
may request that the embedded data be sent to itself while 
the main portion of signal 90 or data 95 is transmitted to 
the TV 80. For example, a broadcast TV signal 85 may 
contain VBI-embedded data that is related to the program 
being broadcast, such as biographies of the actors, 
information about upcoming episodes, historical facts, etc. 
The user could then browse the VBI-embedded data on the 
remote control 10 while viewing the program on the primary 
viewing screen of the TV 80. If the VBI-embedded data (or 
digitally embedded data from a digital signal) is HTML data, 
the remote control 10 may display options for the user to 
respond to program-related issues, for example by voting on 
what should happen to certain characters, or whether the 
episode was good or bad. The user could then respond by 
using the appropriate buttons on the remote control 10 (as 
indicated on the display 15) , which would be programmed to 
send appropriate information to an associated web site. Or 
the embedded data may actually be a web page providing 
access to a web site related to the program being broadcast, 
in which case the remote control 10 could access the web 
site through the base station 75 to gain more detailed 
information related to the program. Of course, the embedded 
data does not have to be related to the broadcast program, 
but a likely use of the embedded data is that it would be so 
related. Another use for embedded data is to send lists of 
web sites associated with advertisements being broadcast. 
The user could then browse the list on the remote control's 
display 15 during or after the advertisements, and tag or 
bookmark them for later reference. There are many other 
uses of embedded data, and the foregoing are intended to be 
only illustrative, and in no way limiting of the types of 
data the present invention is designed to make use of. 

The previous discussions have presumed for the most 
part that the base station 75 was physically separated from 
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the TV 80 and from the remote control 10. It should be 
appreciated that in an embodiment where the base station 75 
is physically integrated with the TV 80, the associated 
communication channel (indicated by arrow 105) would not be 
5 required. Similarly, if the base station 75 is physically 
integrated with the remote control 10, there is no need for 
the base station 75 to "send" video signals to the remote 
control 10, and the associated communication channels 
(indicated by arrows 110 and 115) would not be required. In 

10 the latter case, the RF communications hardware could then 
be eliminated, producing additional cost savings, size 
reduction, maintenance benefits, etc. 

The hardware of a typical base station 75 will now be 
described, as seen in FIG. 3. The hardware is capable of 

15 receiving various data streams (signals 85 and/or other data 
95) , analog and/or digital, and of RF encoding the data 
streams and transmitting them for receipt by an RF receiver 
(e.g., one in the remote control 10). The hardware is also 
capable of processing and passing along data streams to both 

20 analog TV inputs and digital TV inputs as indicated by 
arrows 255 and 260 respectively. 

Signals 85 may be analog or digital. Analog signals 
will enter the base station's "tuner + analog to digital 
converter" 125, and "analog switching and MUX box" 265. 

25 Digital signals enter the base station 75 through USB port 
130, or through other digital inputs capable of handling 
higher-bandwidth data. Other data 95 (including HTML data) 
may enter the base station 75 through various ports 130, 
140, and 145, as is described herein, and even through IrDA 

30 port 135, though that is not preferred. The incoming data 
85 and/or 95 may be transferred to the data bus 150, but 
that is not necessary, as data passing through the switching 
and MUX box 265, for example, may be passed directly to an 
analog TV input. Also, data '95 entering through ports 130, 

35 135, 140, and/or 145 first passes, through the UART 
(Universal Asynchronous Receiver and Transmitter) 160 in the 
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IO-ASIC (Input Output - Application Specific Integrated 
Circuit ) 165. 

Analog data 85 to be sent to the remote control 10 
passes through "tuner and analog to digital converter" 125, 
which converts the data to digital data and tunes to a 
particular channel for transmitting to the remote control 10 
based on commands sent to the base station 75 from the 
remote control 10. The data 85 may then be processed by CPU 
155, such as perhaps to strip out only embedded data to pass 
along to the remote control 10, or to create an alternate 
view of a virtual world for a video game application. 
Converted data 85 is, sent to an MPEG A/V encoder 170, 
whether or not the CPU performed any processing upon it. 

Digital data 85 and/or 95, or portions thereof to be 
sent to the remote control 10 may or may not pass through 
the MPEG A/V encoder 170, since data such as text will not 
necessarily require the MPEG compressing as video data does. 

Data 85 and/or 95 or portions thereof, whether 
originating from analog or digital data streams, then pass 
through MUX 175, either directly or after passing through 
MPEG A/V encoder 170 as stated. All data entering MUX 175 
at that point is digital data. MUX 175 combines MPEG A/V 
data streams with other data streams such as HTML data, then 
sends the combined data stream to an RF encoder 180 which 
passes the data through an antenna 185 so the data may be 
transmitted to the remote control 10. The HTML data sent to 
MUX 175 may be only part of the HTML data stream 85 and/or 
95 originally entering the base station 75, as CPU 155 may 
first process data 85 and/or 95 and determine, based on 
requests from the remote control 10, that some of it should 
be passed to TV 80 instead. 

Data 85 and/or 95 or portions thereof to be sent to an 
analog TV input are treated differently. Analog data is 
sent through the switching and MUX box 265 to be passed to 
the analog TV input, or to the analog to digital converter 
125 if necessary. Switching and MUX box 265 determines 



WO 99/34599 



PCT/US98/27415 



25 



which of multiple possible input signals to send to the 
input or inputs of an analog TV, based on commands from the 
remote control 10. If the analog TV has as many inputs as 
there are analog signals, and if the base station 75 has 
5 enough outputs connecting thereto, then the switching and 
MUX box 265 may send each of the analog signals to the 
analog TV. Before being sent to the analog TV, the analog 
data may undergo further processing as herein described, in 
which case the data typically would have to have first been 

10 converted to digital data. 

Digital data 85 and/or 95 or portions thereof to be 
sent to an analog TV input may also be processed by CPU 155, 
but that is not required. And before being sent to 
switching and MUX box 265 for transmission to the analog TV 

15 input, the data 85 and/or 95 or portions thereof must first 
undergo NTSC or PAL conversion, as indicated by NTSC/PAL 
encoder 270. 

Data 85 and/or 95 or portions thereof to be sent to a 
digital TV input are also treated differently. Digital data 

20 85 and/or 95 enters through USB port 130. or other digital 
port, and may be processed by CPU 155 just as the other data 
streams. Analog data is processed if necessary, after 
passing through the "tuner + analog to digital converter" 
125. The data and/or portions thereof are then transmitted 

25 to a digital TV input back through USB port 130 or other 
digital port. 

Turning to other specific hardware components, the data 
bus 150 also connects an RTC (real-time clock) 190, memory 
chips 195, 200, 205, and 210, the IO-ASIC 165, and port 130, 

30 which is a USB port. The other communications ports (135, 
140, 145, and control IR port 215), are connected to the IO- 
ASIC 165 as described herein. The base station 75 also has 
an alphanumeric display 220 for displaying status messages 
and the like. Power is supplied by an external power source 

35 225. 
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The RTC 150 is preferably a crystal-based clock, and is 
used to synchronize IR commands with the proper TV 
broadcast, radio broadcast, satellite broadcast, or other 
signals to satisfy the consumer's requests. In addition, 
5 the RTC 150 may be used to determine when a particular title 
or titles from a CD have completed based on the play-time 
associated with the title or titles. 

Memory 195 is a boot ROM for holding the system 
software, and memory 210 is a flash ROM for storing 
10 downloaded information such as IR command libraries of 
devices being controlled, or other information such as data 
related to the user's CD collection. The flash ROM 210 is 
preferably at least 2-8 Megabytes. Memory chips 200 and 205 
are RAM chips used for working memory. Memory 200 is shown 
15 as a DRAM, and is preferably at least 8 megabytes. Memory 
205 is shown as an SRAM and is preferably at least 256-512 
Kilobytes. The SRAM 620 may be used for certain operations 
such as MPEG encoding and/or decoding that require faster 
speed than other operations. 
20 The IO-ASIC 165 comprises UART 160, a PWM (Pulse Width 

Modulator) 230, a 32-channel ICU (Interrupt Controller Unit) 
235, a 2-bank DRAMC (DRAM controller) 240, an LCD controller 
245, and its own 16-bit 4-channel timer 250. The UART 160 
is used to receive data through control IR port 215, and to 
25 transmit and receive data through ports 135, 140, and 145. 
Control IR port 215 or its equivalent is essential because 
the data received through control IR port 215 are the IR 
commands from the remote control 10. The other ports (130, 
135, 140, and 145) are optional, but at least one is 
required in order for the base station 75 to connect to an 
outside data source such as the internet. Port 135 is an 
IrDA (Infrared Data Association) port used for wireless 
communication. Port 14 0 is an RS232 port for connection to 
a standard serial port of a PC. Port 145 is a modem port 
35 for connection directly to telephone lines. Port 130 is a 
USB port which is an industry standard port for "plug and 
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play" connection to PCs. The USB port 130 is connected 
directly to bus 150 and can transfer data at a rate of up to 
4 megabytes per second, much faster than the UART 160 which 
can typically transfer data at a rate of only up to about 
5 115 kilobytes per second. The ports other than the USB port 
130 are connected to bus 150 only through the UART 160 in 
the IO-ASIC 165. 

The PWM 230 may be used to activate a speaker (not 
shown) to alert the user of errors on input when 

10 programming, or attempted unauthorized access to certain 
data, or a pending request, or any other event through the 
use of various beeping or buzzing sounds of various 
frequencies, volumes, and/or durations. Voice data may also 
be employed where the voice data is either predetermined, or 

15 dynamically loaded into the base station 75, or dynamically 
created by suitable software. The ICU 235 coordinates and 
controls requests to the CPU 155 from various input and 
output devices either integrated with the base station 75 or 
connected thereto. The DRAMC 240 controls access to the 

20 DRAM 200, and LCD controller 24 5 controls what is displayed 
on the alphanumeric display 220. The timer 250 is used for 
short timing (on the order of milliseconds) within the IO- 
ASIC 165 (such as refreshing the alphanumeric display 220, 
etc. ) . 

25 The hardware of a typical remote control 10 will now be 

described, as seen in FIG. 4. The remote control 10 - is 
capable of receiving RF data 275 through an RF antenna 280. 
The data 275 is transmitted from the base station's antenna 
185 (FIG. 3), and, may include audio, video, and or other 

30 data such as HTML data. The data 275 passes first through a 
"transport Demux and MPEG A/V decoder" (Demux) 285 where any 
audio data 290, video data 300, and HTML or other data 305 
is separated. 

A data bus 310 connects an RTC ■ (real-time clock) 315, 
35 CPU 320, memory chips 325, 330, 335, and 340, IO-ASIC 345, 
and USB port 350. Various other communications ports 355, 
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360, 365, and 370, are connected to the IO-ASIC 345 as will 
be described shortly. The remote control has a touch screen 
375 and LCD 380, and may also have a speaker 385 for sound. 
The power source 390 is shown preferably as a 9V battery or 
9V from an external power source. 

The RTC 315 is preferably a crystal-based clock, and is 
used to synchronize IR commands with the proper TV 
broadcast, radio broadcast, satellite broadcast, or other 
signals to satisfy the consumer's requests. In addition, 
the RTC 315 is used to determine when a particular title or 
titles from a CD have completed based on the play-time 
associated with the title or titles. 

Memory 325 is a boot ROM for holding the system 
software. Memory 340 is a flash ROM, and stores downloaded 
information such as TV schedules, CD track data, pre-loaded 
IR command libraries, etc. The flash ROM 340 is preferably 
at least 2-8 Megabytes. Memory chips 330 and 335 are RAM 
chips used for working memory. Memory 330 is shown as a 
DRAM, and is preferably at least 8 megabytes. Memory 335 is 
20 shown as an SRAM and is preferably at least 256-512 
Kilobytes. The SRAM 335 is used for certain operations such 
as video encoding and/or decoding that may require faster 
speed than other operations. 

The IO-ASIC 345 comprises a UART (Universal 
25 Asynchronous Transmitter and Receiver) 395, a 32-channel ICU 
(Interrupt Controller Unit) 400, a 2-bank DRAMC (DRAM 
controller) 405, an 8-channel CSC (Chip Select Controller) 
410, an LCD controller 415, its own 16-bit 4-channel timer 
420, and a PWM (Pulse Width Modulator) 425. The UART 395 is 
used to transmit data through port 355 which is a control IR 
port, and to transmit and receive data through ports 360, 
365, and 370. Control IR port 355 or its equivalent is 
essential because the data sent through control IR port 355 
are the IR commands to control devices. The other ports 
35 (320, 360, 365, and 370) are optional, but at least one is 
required in order for the remote control 10 to connect to an 
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outside data source such as the internet. Port 360 is an 
IrDA port used for wireless communication. Port 365 is an 
RS232 port for connection to a standard serial port of a PC. 
Port 370 is a modem port for connection directly to 
5 telephone lines. Port 350 is a USB port which is an 
industry standard port for "plug and play" connection to 
PCs. The USB port 350 is connected directly to bus 310 and 
can transfer data at a rate of up to 4 megabytes per second, 
much faster than the UART 395 which can typically transfer 

10 data at a rate of only up to about 115 kilobytes per second. 
The ports other than the USB port 350 are connected to bus 
310 only through the UART 395 in the IO-ASIC 345. 

The ICU 400 coordinates and controls requests to the 
CPU 320 from various input and output devices either 

15 integrated with the remote control 10 or connected thereto. 
The DRAMC 405 controls access to the DRAM 330, and the CSC 
410 processes the consumer's touch screen 375 requests, 
which first pass through a 10-bit analog-to-digital 
converter 430. The CSC 410 also interacts with the flash 

20 ROM 340, the modem port 370, and other components, and 
serves a fairly general control function in the 10 
subsystem. The LCD controller 415 controls what is 
displayed on the LCD display 380. The timer 420 is used for 
short timing (on the order of milliseconds) within the 10- 

25 ASIC 345 (such as refreshing the LCD display 380) . The PWM 
425 sends digital signals to a digital to analog amplifier 
435 which passes the signal to speaker 385. The speaker 385 
may be used to alert the consumer of errors on input when 
programming, or attempted unauthorized access to certain 

30 data, or a pending request, or any other event through the 
use of various beeping or buzzing sounds of various 
frequencies, volumes, and/or durations. Or the speaker 385 
may be used for voice sound as previously described. 

While certain embodiments are' illustrated in the 

35 drawings and have just been described herein, it will be 
apparent to those skilled in the art that many modifications 
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can be made to the embodiments without departing from the 
inventive concepts described. Accordingly, the invention is 
not to be restricted except by the claims which follow. 
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Claims 

1 . A method of viewing multiple data streams 
substantially simultaneously comprising the steps of: 

providing a base station; 
5 providing a first device having a display, the 

first device being capable of displaying full motion video; 

providing a second device having a display, the 
second device being capable of displaying full motion video; 

receiving a first data stream into the base 

10 station; 

receiving a second data stream into the base 

station; 

transmitting the first data stream from the base 
station to the first device; 
15 displaying the first data stream on the display of 

the first device; 

transmitting the second data stream from the base 
station to the second device; and 

displaying the second data stream on the display 
20 of the second device. 

2. The method of claim 1 wherein the first data 
stream comprises a broadcast TV signal. 

3. The method of claim 1 wherein the first data 
stream comprises a satellite TV signal. 

25 4. The method of claim 1 wherein the first data 

stream comprises HTML data. 

5. The method of claim 4 wherein the HTML data enters 
the base station from the internet. 

6. The method of claim 1 wherein the first device is 

30 a TV. 
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7. The method of claim 6 wherein the second device is 
a hand-held remote control. 

8. The method of claim 7 wherein the second data 
stream is transmitted from the base station to the hand-held 

5 remote control using radio frequency transmission. 



10 



of: 



9. The method of claim 1 further comprising the steps 

discontinuing to display the first data stream on 
the display of the first device; 

displaying the second data stream on the display 
of the first device; 

discontinuing to display the second data stream on 
the display of the second device; and 

displaying the first data stream on the display of 
15 the second device. 

10. A method of viewing different portions of data 
from a data stream on two physically separate displays 
substantially simultaneously comprising the steps of: 
providing a base station; 
20 receiving the data stream into the base station; 

processing the data stream within the base station 
to separate the data stream into a first portion and a 
second portion; 

transmitting the first portion from the base 
25 station to the first display; 

displaying the first portion on the first display; 
transmitting the second portion from the base 
station to the second display; and 

displaying the second portion on the second 

30 display. 



11. The method of claim 10 wherein the data stream 
comprises an analog signal having a vertical blanking 
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interval with data embedded within the vertical blanking 
interval, and wherein the second portion comprises data in 
the vertical blanking interval. 

12. The method of claim 11 wherein the data stream 
comprises a broadcast TV signal. 

13. The method of claim 11 wherein the data stream 
comprises a satellite transmission signal. 

14. The method of claim 11 wherein the second portion 
comprises HTML data. 

15. The method of claim 10 wherein the data stream 
comprises a digital signal. 

16. The method of claim 10 wherein the second display 
is on a hand-held remote control. 

17. The method of claim 10 wherein the first display 
is on an internet-enabled TV. 

18. The method of claim 10 further comprising the 
steps of: 

discontinuing to display the first portion on the 
first display; 

displaying the second portion on the first 

display; 

discontinuing to display the second portion on the 
second display; and 

displaying the first portion on the second 

display. 

19. A method of viewing multiple channels of a TV 
broadcast signal substantially simultaneously comprising the 
steps -of: 
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20 



25 



'30 



display; 



station; 



providing a base station having a tuner; 

providing a first device having a tuner and a 

providing a second device having a display; 
receiving the TV broadcast . signal into the base 



sending the TV broadcast signal from the base 
station to the first device; 

tuning the first device to a predetermined channel 
using the tuner of the first devices- 
displaying the channel of the TV broadcast signal 
tuned to by the tuner of the first device on the display of 
the first device; 

tUn±ng the base station to a predetermined channel 
15 using the tuner of the base station; 

sending the channel of the TV broadcast signal 
tuned to by the tuner of the base station from the base 
station to the second device; and 

displaying the channel of the TV broadcast signal 
sent to the second device on the display of the second 
device . 

20. The method of claim 19 wherein the second device 
is a portable hand-held device. 

21. The method of claim 19 wherein the channel of the 
TV broadcast signal sent from the base station to the second 
device is sent using radio frequency transmission. 

22. A system for providing enhanced viewing of 
multiple data streams comprising: 

a first device having a display, the first device 
being capable of displaying full motion video; 

a second device having a display, the second 
device being capable of displaying full motion video, and 
having communications hardware for sending control infrared 
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(IR) commands, and a radio frequency (RF) antenna for 
receiving RF transmissions; and 

a base station connected to the first device by a 
cable, and having communications hardware for receiving 
control IR commands and for sending RF transmissions. 

23. The system of claim 22 wherein the second device 
is a hand-held remote control . 

24. The system of claim 23 wherein the remote control 
further comprises a USB port. 

25. The system of claim 22 wherein the first device 
has communications hardware for receiving control IR 
commands . 

26. The system of claim 22 wherein the base station 
further comprises a tuner. 

27. The system of claim 23 wherein the remote control 
comprises a tuner. 

28. A method of swapping channels from a display on a 
first device to a display on a second device, and from the 
display on the second device to the display on the first 
device, wherein the channels are both at least initially 
contained within a single data stream, and wherein the first 
device has communications hardware for transmitting infrared 
(IR) commands and the first device is receiving the channel 
on its display via RF communications from a base station, 
and the second device has communications hardware for 
receiving IR commands, comprising the steps: 

actuating a button on the first device, said 
actuation causing an infrared (IR) command sequence to be 
sent from the first device to the second device commanding 
the second device to adjust a tuner therein to a 
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predetermined channel, said actuation also causing another 
IR command sequence to be sent from the first device to the 
base station commanding the base station to adjust a tuner 
contained therein to a predetermined channel. 

29. The method of claim 28 wherein the first device is 
a hand-held remote control, and the second device is a TV. 

30. The method of claim 29 wherein the remote control 
has a memory and wherein the remote control stores in the 
memory information related to the IR commands sent such that 
the remote control is aware of the channels tuned to by the 
tuners. 



31. The method of claim 7 wherein the second data 
stream is transmitted from the base station to the remote 
control using Firewire 1394 protocol wired communication. 

32. The method of claim 15 wherein the digital signal 
conforms to a protocol wherein there are dedicated slots for 
video data, audio data, and other data, and wherein the 
second portion comprises the other data. 

33. The method of claim 32 wherein the protocol is 
MPEG and the other data is HTML data. 

34. The method of claim 16 wherein the second portion 
is transmitted from the base station to the second display 
via Firewire 1394 protocol wired communication. 

35. The system of claim 23 wherein the remote control 
further comprises a Firewire 1394 port. 

36. The method of claim 29 wherein the remote control 
sends information to the base station via RF communications. 
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